Timezone: »
People easily recognize new visual categories that are new combinations of known components. This compositional generalization capacity is critical for learning in real-world domains like vision and language because the long tail of new combinations dominates the distribution. Unfortunately, learning systems struggle with compositional generalization because they often build on features that are correlated with class labels even if they are not "essential" for the class. This leads to consistent misclassification of samples from a new distribution, like new combinations of known components.
Here we describe an approach for compositional generalization that builds on causal ideas. First, we describe compositional zero-shot learning from a causal perspective, and propose to view zero-shot inference as finding "which intervention caused the image?". Second, we present a causal-inspired embedding model that learns disentangled representations of elementary components of visual objects from correlated (confounded) training data. We evaluate this approach on two datasets for predicting new combinations of attribute-object pairs: A well-controlled synthesized images dataset and a real world dataset which consists of fine-grained types of shoes. We show improvements compared to strong baselines.
Author Information
Yuval Atzmon (NVIDIA Research)
Felix Kreuk (Bar-Ilan University)
Uri Shalit (Technion)
Gal Chechik (NVIDIA, BIU)
Related Events (a corresponding poster, oral, or spotlight)
-
2020 Poster: A causal view of compositional zero-shot recognition »
Wed Dec 9th 05:00 -- 07:00 PM Room Poster Session 3
More from the Same Authors
-
2021 : Bandits with Partially Observable Confounded Data »
Guy Tennenholtz · Uri Shalit · Shie Mannor · Yonathan Efroni -
2021 : Covariate Shift of Latent Confounders in Imitation and Reinforcement Learning »
Guy Tennenholtz · Assaf Hallak · Gal Dalal · Shie Mannor · Gal Chechik · Uri Shalit -
2021 : Covariate Shift of Latent Confounders in Imitation and Reinforcement Learning »
Guy Tennenholtz · Assaf Hallak · Gal Dalal · Shie Mannor · Gal Chechik · Uri Shalit -
2021 : Covariate Shift of Latent Confounders in Imitation and Reinforcement Learning »
Guy Tennenholtz · Assaf Hallak · Gal Dalal · Shie Mannor · Gal Chechik · Uri Shalit -
2021 : Uri Shalit - Calibration, out-of-distribution generalization and a path towards causal representations »
Uri Shalit -
2021 Poster: Personalized Federated Learning With Gaussian Processes »
Idan Achituve · Aviv Shamsian · Aviv Navon · Gal Chechik · Ethan Fetaya -
2021 Poster: Improve Agents without Retraining: Parallel Tree Search with Off-Policy Correction »
Gal Dalal · Assaf Hallak · Steven Dalton · iuri frosio · Shie Mannor · Gal Chechik -
2020 Poster: Identifying Causal-Effect Inference Failure with Uncertainty-Aware Models »
Andrew Jesson · Sören Mindermann · Uri Shalit · Yarin Gal -
2018 : Accepted papers »
Sven Gowal · Bogdan Kulynych · Marius Mosbach · Nicholas Frosst · Phil Roth · Utku Ozbulak · Simral Chaudhary · Toshiki Shibahara · Salome Viljoen · Nikita Samarin · Briland Hitaj · Rohan Taori · Emanuel Moss · Melody Guan · Lukas Schott · Angus Galloway · Anna Golubeva · Xiaomeng Jin · Felix Kreuk · Akshayvarun Subramanya · Vipin Pillai · Hamed Pirsiavash · Giuseppe Ateniese · Ankita Kalra · Logan Engstrom · Anish Athalye -
2018 Poster: Mapping Images to Scene Graphs with Permutation-Invariant Structured Prediction »
Roei Herzig · Moshiko Raboh · Gal Chechik · Jonathan Berant · Amir Globerson -
2018 Poster: Removing Hidden Confounding by Experimental Grounding »
Nathan Kallus · Aahlad Puli · Uri Shalit -
2018 Spotlight: Removing Hidden Confounding by Experimental Grounding »
Nathan Kallus · Aahlad Puli · Uri Shalit -
2016 : CV @ Scale Challenges »
Manohar Paluri · Gal Chechik -
2016 Workshop: Large Scale Computer Vision Systems »
Manohar Paluri · Lorenzo Torresani · Gal Chechik · Dario Garcia · Du Tran -
2014 Workshop: Analyzing the omics of the brain »
Michael Hawrylycz · Gal Chechik · Mark Reimers -
2010 Spotlight: Online Learning in The Manifold of Low-Rank Matrices »
Uri Shalit · Daphna Weinshall · Gal Chechik -
2010 Poster: Online Learning in The Manifold of Low-Rank Matrices »
Uri Shalit · Daphna Weinshall · Gal Chechik -
2009 Workshop: Machine Learning in Computational Biology »
Gal Chechik · Tomer Hertz · William S Noble · Yanjun Qi · Jean-Philippe Vert · Alexander Zien -
2009 Mini Symposium: Machine Learning in Computational Biology »
Yanjun Qi · Jean-Philippe Vert · Gal Chechik · Alexander Zien · Tomer Hertz · William S Noble -
2009 Poster: An Online Algorithm for Large Scale Image Similarity Learning »
Gal Chechik · Uri Shalit · Varun Sharma · Samy Bengio -
2008 Workshop: Machine Learning in Computational Biology »
Gal Chechik · Christina Leslie · Quaid Morris · William S Noble · Gunnar Rätsch -
2008 Mini Symposium: Machine Learning in Computational Biology »
Gal Chechik · Christina Leslie · Quaid Morris · William S Noble · Gunnar Rätsch -
2007 Workshop: Machine Learning in Computational Biology (Part 2) »
Gal Chechik · Christina Leslie · Quaid Morris · William S Noble · Gunnar Rätsch · Koji Tsuda -
2007 Workshop: Machine Learning in Computational Biology (Part 1) »
Gal Chechik · Christina Leslie · Quaid Morris · William S Noble · Gunnar Rätsch · Koji Tsuda -
2006 Workshop: New Problems and Methods in Computational Biology »
Gal Chechik · Quaid Morris · Koji Tsuda · Gunnar Rätsch · Christina Leslie · William S Noble -
2006 Poster: Max-margin classification of incomplete data »
Gal Chechik · Geremy Heitz · Gal Elidan · Pieter Abbeel · Daphne Koller -
2006 Poster: Temporal and Cross-Subject Probabilistic Models for fMRI Prediction Task »
Alexis Battle · Gal Chechik · Daphne Koller -
2006 Spotlight: Max-margin classification of incomplete data »
Gal Chechik · Geremy Heitz · Gal Elidan · Pieter Abbeel · Daphne Koller -
2006 Talk: Temporal and Cross-Subject Probabilistic Models for fMRI Prediction Task »
Alexis Battle · Gal Chechik · Daphne Koller