Workshop
Compositional Learning: Perspectives, Methods, and Paths Forward
Ying Wei 路 Jonathan Richard Schwarz 路 Yilun Du 路 Laurent Charlin 路 Mengye Ren 路 Matthias Bethge
Sun 15 Dec, 8:25 a.m. PST
Compositional learning, inspired by the human ability to derive complex ideas from simpler constituents, seeks to equip machines with analogous capabilities for understanding, reasoning, and adaptive learning. This methodology bolsters machines' ability to generalize to out-of-distribution samples through the recombination of learned components, proving effective across diverse tasks such as machine translation, visual reasoning, image generation, reinforcement learning, and more. Despite notable advancements, persistent challenges remain in achieving robust compositional generalization and reasoning within even the most advanced foundation models. Our workshop aims to discuss these challenges as well as untapped opportunities ahead from the following four aspects: exploring the capacity for compositionality in foundation models and dissecting the underlying mechanisms of their compositional learning; devising reliable and model-agnostic strategies for constructing compositional systems; establishing theoretical and empirical connections between modular architectures and compositional generalization; and extending compositional learning principles to continual learning contexts. By confronting these themes, we aim to foster a collaborative exploration of theoretical and empirical dimensions of compositional learning, thus advancing understanding and practical applications of compositional learning.
Schedule
Sun 8:25 a.m. - 8:30 a.m.
|
Introduction and Opening Remarks
(
Intro
)
>
SlidesLive Video |
Ying Wei 馃敆 |
Sun 8:30 a.m. - 9:00 a.m.
|
Colin Raffle
(
Invited Talk
)
>
SlidesLive Video |
馃敆 |
Sun 9:00 a.m. - 9:30 a.m.
|
Ranjeev Alur
(
Invited Talk
)
>
SlidesLive Video |
馃敆 |
Sun 10:30 a.m. - 10:45 a.m.
|
Instruct-SkillMix: A Powerful Pipeline for LLM Instruction Tuning.
(
Contributed Talk
)
>
SlidesLive Video |
馃敆 |
Sun 10:45 a.m. - 11:00 a.m.
|
Provably Learning Concepts by Comparison
(
Contributed Talk
)
>
SlidesLive Video |
馃敆 |
Sun 11:00 a.m. - 11:30 a.m.
|
Irina Rish
(
Invited Talk
)
>
|
馃敆 |
Sun 11:30 a.m. - 12:00 p.m.
|
Jacob Andreas
(
Invited Talk
)
>
SlidesLive Video |
馃敆 |
Sun 1:30 p.m. - 2:00 p.m.
|
Chuang Gan
(
Invited Talk
)
>
SlidesLive Video |
馃敆 |
Sun 2:00 p.m. - 2:30 p.m.
|
Thomas Kipf
(
Invited Talk
)
>
SlidesLive Video |
馃敆 |
Sun 3:30 p.m. - 3:45 p.m.
|
Scalable and Interpretable Quantum Natural Language Processing: An Implementation on Trapped Ions
(
Contributed Talk
)
>
SlidesLive Video |
馃敆 |
Sun 3:45 p.m. - 4:00 p.m.
|
Successes and Limitations of Object-centric Models at Compositional Generalisation
(
Contributed Talk
)
>
SlidesLive Video |
馃敆 |
Sun 4:00 p.m. - 5:00 p.m.
|
Panel Discussion
(
Panel
)
>
SlidesLive Video |
馃敆 |
-
|
Enhancing Generalization in Sparse Mixture of Experts Models: The Case for Increased Expert Activation in Compositional Tasks ( Poster ) > link | Jinze Zhao 路 Junjie Yang 路 Peihao Wang 路 Yingbin Liang 路 Zhangyang "Atlas" Wang 馃敆 |
-
|
Geometric Signatures of Compositionality in Language Models ( Poster ) > link | Thomas Jiralerspong 路 Jin Hwa Lee 路 Lei Yu 路 Emily Cheng 馃敆 |
-
|
A Linear Network Theory of Iterated Learning ( Poster ) > link | Devon Jarvis 路 Richard Klein 路 Benjamin Rosman 路 Andrew Saxe 馃敆 |
-
|
Compositional Risk Minimization ( Poster ) > link | Divyat Mahajan 路 Mohammad Pezeshki 路 Ioannis Mitliagkas 路 Kartik Ahuja 路 Pascal Vincent 馃敆 |
-
|
Transformer-based Imagination with Slot Attention ( Poster ) > link | Yosuke Nishimoto 路 Takashi Matsubara 馃敆 |
-
|
Crafting Global Optimizers to Reasoning Tasks via Algebraic Objects in Neural Nets ( Poster ) > link | Yuandong Tian 馃敆 |
-
|
Scalable and interpretable quantum natural language processing: an implementation on trapped ions ( Oral ) > link | Tiffany Duneau 路 Saskia Bruhn 路 Gabriel Matos 路 Tuomas Laakkonen 路 Katerina Saiti 路 Anna Pearson 路 Konstantinos Meichanetzidis 路 Bob Coecke 馃敆 |
-
|
Relational composition during attribute retrieval in GPT is not purely linear ( Poster ) > link | Michael McCoy 路 Anna Leshinskaya 馃敆 |
-
|
Object-Centric Temporal Consistency via Conditional Autoregressive Inductive Biases ( Poster ) > link | Cristian Meo 路 Akihiro Nakano 路 Mircea Lic膬 路 Aniket Didolkar 路 Masahiro Suzuki 路 Anirudh Goyal 路 Mengmi Zhang 路 Justin Dauwels 路 Yutaka Matsuo 路 Yoshua Bengio 馃敆 |
-
|
Faster Slot Decoding using Masked Transformer ( Poster ) > link | Akihiro Nakano 路 Masahiro Suzuki 路 Yutaka Matsuo 馃敆 |
-
|
CoS: Enhancing Personalization with Context Steering ( Poster ) > link | Sashrika Pandey 路 Jerry He 路 Mariah Schrum 路 Anca Dragan 馃敆 |
-
|
Provably Learning Concepts by Comparison ( Oral ) > link | Yujia Zheng 路 Shaoan Xie 路 Kun Zhang 馃敆 |
-
|
OC-CLIP : Object-centric Binding in Contrastive Language-Image Pretraining ( Poster ) > link | Rim Assouel 路 Pietro Astolfi 路 Florian Bordes 路 Michal Drozdzal 路 Adriana Romero 馃敆 |
-
|
Exploring A Bayesian View On Compositional and Counterfactual Generalization ( Poster ) > link | Patrik Reizinger 路 Rahul Krishnan 馃敆 |
-
|
Towards Object-Centric Learning with General Purpose Architectures ( Poster ) > link | Jack Brady 路 Julius von K眉gelgen 路 S茅bastien Lachapelle 路 Simon Buchholz 路 Thomas Kipf 路 Wieland Brendel 馃敆 |
-
|
Generating Intermediate Representations for Compositional Text-To-Image Generation ( Poster ) > link | Ran Galun 路 Sagie Benaim 馃敆 |
-
|
Diffusion Beats Autoregressive: An Evaluation of Compositional Generation in Text-to-Image Models ( Poster ) > link | Arash Mari Oriyad 路 Rezaei 路 Mahdieh Soleymani 路 Mohammad Hossein Rohban 馃敆 |
-
|
From Text to Pose to Image: Improving Diffusion Model Control and Quality ( Poster ) > link | Cl茅ment Bonnet 路 Ariel Lee 路 Franck Wertel 路 Antoine Tamano 路 Tanguy Cizain 路 Pablo Ducru 馃敆 |
-
|
Understanding Simplicity Bias towards Compositional Mappings via Learning Dynamics ( Poster ) > link | Yi Ren 路 Danica J. Sutherland 馃敆 |
-
|
GSR-Bench: A Benchmark for Grounded Spatial Reasoning Evaluation via Multimodal LLMs ( Poster ) > link | Navid Rajabi 路 Jana Kosecka 馃敆 |
-
|
Latent Concept-based Explanation of NLP Models ( Poster ) > link | Xuemin Yu 路 Fahim Dalvi 路 Nadir Durrani 路 Marzia Nouri 路 Hassan Sajjad 馃敆 |
-
|
Pretraining Frequency Predicts Compositional Generalization of CLIP on Real-World Tasks ( Poster ) > link | Thadd盲us Wiedemer 路 Yash Sharma 路 Ameya Prabhu 路 Matthias Bethge 路 Wieland Brendel 馃敆 |
-
|
A Multimodal Chain of Tools for Described Object Detection ( Poster ) > link | Kwanyong Park 路 Youngwan Lee 路 Yong-Ju Lee 馃敆 |
-
|
Crosslingual Capabilities and Knowledge Barriers in Multilingual Large Language Models ( Poster ) > link | Lynn Chua 路 Badih Ghazi 路 Yangsibo Huang 路 Pritish Kamath 路 Ravi Kumar 路 Pasin Manurangsi 路 Amer Sinha 路 Chulin Xie 路 Chiyuan Zhang 馃敆 |
-
|
Compositional Communication with LLMs and Reasoning about Chemical Structures ( Poster ) > link | Sarathkrishna Swaminathan 路 Dmitry Zubarev 馃敆 |
-
|
Unraveling the Latent Hierarchical Structure of Language and Images via Diffusion Models ( Poster ) > link | Antonio Sclocchi 路 Noam Levi 路 Alessandro Favero 路 Matthieu Wyart 馃敆 |
-
|
Evaluating Language Models Planning Capabilities on Goal Ordering Challenges ( Poster ) > link | Eran Hirsch 路 Guy Uziel 路 Ateret Anaby Tavor 馃敆 |
-
|
An Integrated Approach to Open-World Compositional Zero-Shot Learning ( Poster ) > link | Hirunima Jayasekara 路 Khoi Pham 路 Nirat Saini 路 Abhinav Shrivastava 馃敆 |
-
|
Sometimes I am a Tree: Data Drives Fragile Hierarchical Generalization ( Poster ) > link | Tian Qin 路 Naomi Saphra 路 David Alvarez-Melis 馃敆 |
-
|
Can Models Learn Skill Composition from Examples? ( Poster ) > link | Haoyu Zhao 路 Simran Kaur 路 Dingli Yu 路 Anirudh Goyal 路 Sanjeev Arora 馃敆 |
-
|
Rule Extrapolation in Language Models: A Study of Compositional Generalization on OOD Prompts ( Poster ) > link | Anna M茅sz谩ros 路 Szilvia Ujv谩ry 路 Wieland Brendel 路 Patrik Reizinger 路 Ferenc Huszar 馃敆 |
-
|
Dynamic Symbolic Representation and LLM to Enhance Task Abstraction in Hierarchical Reinforcement Learning ( Poster ) > link | Sao Mai Nguyen 路 Zihe Ji 路 Mehdi Zadem 馃敆 |
-
|
The role of segmentation in compositional generalisation ( Oral ) > link | Milton Montero 路 Jeffrey Bowers 路 Gaurav Malhotra 馃敆 |
-
|
Learning Via Imagination: Controlled Diffusion Image Augmentation ( Poster ) > link | Judah Goldfeder 路 Patrick Puma 路 Gabriel Guo 路 Gabriel Trigo 路 Hod Lipson 馃敆 |
-
|
Instruct-SkillMix: A Powerful Pipeline for LLM Instruction Tuning ( Oral ) > link | Simran Kaur 路 Simon Park 路 Anirudh Goyal 路 Sanjeev Arora 馃敆 |
-
|
HAMMR : HierArchical MultiModal React agents for generic VQA ( Poster ) > link | Lluis Castrejon 路 Thomas Mensink 路 Howard Zhou 路 Vittorio Ferrari 路 Andre Araujo 路 Jasper Uijlings 馃敆 |
-
|
Can language model plan in extrapolated environments?: Casestudy in textualized Gridworld ( Poster ) > link | Doyoung Kim 路 Jongwon Lee 路 Jinho Park 路 Minjoon Seo 馃敆 |
-
|
Compositional Visual Reasoning with SlotSSMs ( Poster ) > link | Jindong Jiang 路 Fei Deng 路 Gautam Singh 路 Minseung Lee 路 Sungjin Ahn 馃敆 |
-
|
Transformers Can Learn Meta-skills for Task Generalization in In-Context Learning ( Poster ) > link | Ying Fan 路 Steve Yadlowsky 路 Dimitris Papailiopoulos 路 Kangwook Lee 馃敆 |
-
|
Compositional Few-shot Learning of Motions ( Poster ) > link | Omkar Patil 路 Anant Sah 路 Nakul Gopalan 馃敆 |
-
|
ConceptMix: A Compositional Image Generation Benchmark with Controllable Difficulty ( Poster ) > link | Xindi Wu 路 Dingli Yu 路 Yangsibo Huang 路 Olga Russakovsky 路 Sanjeev Arora 馃敆 |