Timezone: »

Object Representations for Learning and Reasoning
William Agnew · Rim Assouel · Michael Chang · Antonia Creswell · Eliza Kosoy · Aravind Rajeswaran · Sjoerd van Steenkiste

Fri Dec 11 08:00 AM -- 07:15 PM (PST) @ None
Event URL: https://orlrworkshop.github.io/ »

Recent advances in deep reinforcement learning and robotics have enabled agents to achieve superhuman performance on a variety of challenging games and learn complex manipulation tasks. While these results are very promising, several open problems remain. In order to function in real-world environments, learned policies must be both robust to input perturbations and be able to rapidly generalize or adapt to novel situations. Moreover, to collaborate and live with humans in these environments, the goals and actions of embodied agents must be interpretable and compatible with human representations of knowledge. Hence, it is natural to consider how humans so successfully perceive, learn, and plan to build agents that are equally successful at solving real world tasks.
There is much evidence to suggest that objects are a core level of abstraction at which humans perceive and understand the world [8]. Objects have the potential to provide a compact, casual, robust, and generalizable representation of the world. Recently, there have been many advancements in scene representation, allowing scenes to be represented by their constituent objects, rather than at the level of pixels. While these works have shown promising results, there is still a lack of agreement on how to best represent objects, how to learn object representations, and how best to leverage them in agent training.
In this workshop we seek to build a consensus on what object representations should be by engaging with researchers from developmental psychology and by defining concrete tasks and capabilities that agents building on top of such abstract representations of the world should succeed at. We will discuss how object representations may be learned through invited presenters with expertise both in unsupervised and supervised object representation learning methods. Finally, we will host conversations and research on new frontiers in object learning.

Fri 8:00 a.m. - 8:15 a.m.
William Agnew
Fri 8:15 a.m. - 9:00 a.m.

Elizabeth Spelke is the Marshall L. Berkman Professor of Psychology at Harvard University and an investigator at the NSF-MIT Center for Brains, Minds and Machines. Her laboratory focuses on the sources of uniquely human cognitive capacities, including capacities for formal mathematics, for constructing and using symbols, and for developing comprehensive taxonomies of objects. She probes the sources of these capacities primarily through behavioral research on human infants and preschool children, focusing on the origins and development of their understanding of objects, actions, people, places, number, and geometry. In collaboration with computational cognitive scientists, she aims to test computational models of infants’ cognitive capacities. In collaboration with economists, she has begun to take her research from the laboratory to the field, where randomized controlled experiments can serve to evaluate interventions, guided by research in cognitive science, that seek to enhance young children’s learning.

Fri 9:02 a.m. - 9:04 a.m.
Learning Object-Centric Video Models by Contrasting Sets (Lightning)   
Fri 9:04 a.m. - 9:06 a.m.
Structure-Regularized Attention for Deformable Object Representation (Lightning)   
Fri 9:06 a.m. - 9:08 a.m.
Learning Long-term Visual Dynamics with Region Proposal Interaction Networks (Lightning)   
Fri 9:08 a.m. - 9:10 a.m.
Self-Supervised Attention-Aware Reinforcement Learning (Lightning)   
Fri 9:10 a.m. - 9:12 a.m.
Emergence of compositional abstractions in human collaborative assembly (Lightning)   
Fri 9:12 a.m. - 9:14 a.m.
Semantic State Representation for Reinforcement Learning (Lightning)   
Fri 9:14 a.m. - 9:16 a.m.
Odd-One-Out Representation Learning (Lightning)   
Fri 9:16 a.m. - 9:18 a.m.
Word(s) and Object(s): Grounded Language Learning In Information Retrieval (Lightning)   
Fri 9:20 a.m. - 9:22 a.m.
Discrete Predictive Representation for Long-horizon Planning (Lightning)   
Fri 9:22 a.m. - 9:24 a.m.
Dynamic Regions Graph Neural Networks for Spatio-Temporal Reasoning (Lightning)   
Fri 9:26 a.m. - 9:28 a.m.
Dexterous Robotic Grasping with Object-Centric Visual Affordances (Lightning)   
Fri 9:28 a.m. - 9:30 a.m.
Understanding designed objects by program synthesis (Lightning)   
Fri 9:29 a.m. - 9:31 a.m.
Learning Embeddings that Capture Spatial Semantics for Indoor Navigation (Lightning)   
Fri 9:30 a.m. - 10:30 a.m.
Poster Session A in GatherTown (Poster Session)  link »
Fri 10:30 a.m. - 11:45 a.m.

How can we obtain object representations in real world environments? How can object representations be applied in robotics? Join us for a panel discussion with Jessica Hamrick, Irina Higgins, Michelle Lee, Josh Tenenbaum, moderated by Klaus Greff.

Jessica Hamrick, Klaus Greff, Michelle A. Lee, Irina Higgins, Josh Tenenbaum
Fri 11:45 a.m. - 12:25 p.m.
Break in GatherTown (Break)  link »
Fri 12:25 p.m. - 12:55 p.m.

Jessica Hamrick is a Senior Research Scientist at DeepMind, where she studies how to build machines that can flexibly build and deploy models of the world. Her work combines insights from cognitive science with structured relational architectures, model-based deep reinforcement learning, and planning. Jessica received a Ph.D. in Psychology from UC Berkeley in 2017, and an M.Eng. and B.S. in Computer Science from MIT in 2012.

Jessica Hamrick
Fri 12:55 p.m. - 1:25 p.m.

Irina Higgins is a research scientist at DeepMind, where she works in the Frontiers team. Her work aims to bring together insights from the fields of neuroscience and physics to advance general artificial intelligence through improved representation learning. Before joining DeepMind, Irina was a British Psychological Society Undergraduate Award winner for her achievements as an undergraduate student in Experimental Psychology at Westminster University, followed by a DPhil at the Oxford Centre for Computational Neuroscience and Artificial Intelligence, where she focused on understanding the computational principles underlying speech processing in the auditory brain. During her DPhil, Irina also worked on developing poker AI, applying machine learning in the finance sector, and working on speech recognition at Google Research.

Irina Higgins
Fri 1:25 p.m. - 1:55 p.m.

Sungjin Ahn is an Assistant Professor of Computer Science at Rutgers University and directs the Rutgers Machine Learning (RUML) lab. He is also affiliated with Rutgers Center for Cognitive Science. His research focus is on how an AI-agent can learn the structure and representations of the world in an unsupervised and compositional way, with a particular interest in object-centric learning. His approach to achieving this is based on deep learning, Bayesian modeling, reinforcement learning, and inspiration from cognitive & neuroscience. He received Ph.D. at the University of California, Irvine with Max Welling and did a postdoc with Yoshua Bengio at Mila. Then, he joined Rutgers University in Fall 2018. He has co-organized ICML 2020 Workshop on Object-Oriented Learning and received the ICML best paper award in ICML 2012.

Sungjin Ahn
Fri 1:55 p.m. - 2:07 p.m.
Contributed Talk : A Symmetric and Object-Centric World Model for Stochastic Environments (Talk)   
Fri 2:07 p.m. - 2:19 p.m.
Contributed Talk : OGRE: An Object-based Generalization for Reasoning Environment (Talk)   
Fri 2:19 p.m. - 2:49 p.m.

Wilka Carvalho is a PhD Candidate in Computer Science at the University of Michigan–Ann Arbor where he is advised by Honglak Lee, Satinder Singh, and Richard Lewis. His long-term research goal is to develop cognitive theories of learning that help us understand how humans infer, reason with, and exploit the rich structure present in realistic visual scenes to enable sophisticated behavioral policies. Towards this end, he is studying how object-centric representation learning and reinforcement learning can bring us closer to human-level artificial intelligence. He is supported by an NSF GRFP Fellowship and a UM Rackham Merit Fellowship.

Wilka Carvalho
Fri 2:49 p.m. - 3:20 p.m.
Break in GatherTown (Break)  link »
Fri 3:20 p.m. - 3:50 p.m.

Renée Baillargeon is an Alumni Distinguished Professor of Psychology at the University of Illinois Urbana-Champaign. Her research examines cognitive development in infancy and focuses primarily on causal reasoning. In particular, she explores how infants make sense of the events they observe, and what explanatory frameworks and learning mechanisms enable them to do so. In addition to this primary focus on causal reasoning, she is interested in a broad range of related issues including object perception, categorization, object individuation, number, and executive-function skills.

Fri 3:50 p.m. - 4:20 p.m.

Object Representations for Robot Manipulation

Reasoning about objects is a fundamental task in robot manipulation. Different representations can have important repercussions on the capabilities and generality of a manipulation system. In this talk I will discuss different ways we represent and reason about objects, ranging from explicit 3D models to raw point clouds.

Fri 4:20 p.m. - 4:32 p.m.
Contributed Talk : Disentangling 3D Prototypical Networks for Few-Shot Concept Learning (Talk)   
Fri 4:32 p.m. - 4:44 p.m.
Contributed Talk : Deep Affordance Foresight: Planning for What Can Be Done Next (Talk)   
Fri 4:44 p.m. - 4:56 p.m.
Contributed talk : Estimating Mass Distribution of Articulated Objects using Non-prehensile Manipulation (Talk)   
Fri 4:56 p.m. - 6:10 p.m.

What should be in an object representation, what should an object representation be able to do, and how do we measure and compare them? Join us for a panel discussion with Wilka Carvalho, Judy Fan, Tejas Kulkarni, and Chris Xie, moderated by Rachit Dubey.

rach0012, Wilka Carvalho, Judith Fan, Tejas Kulkarni, Christopher Xie
Fri 6:10 p.m. - 6:15 p.m.
Concluding Remarks
Fri 6:15 p.m. - 7:15 p.m.
Poster Session B in GatherTown (Poster Session)  link »

Author Information

William Agnew (University of Washington)
Rim Assouel (umontreal.ca)
Michael Chang (University of California, Berkeley)

Ph.D. student at Berkeley AI Research, U.C. Berkeley B.S. in Computer Science from MIT Former research intern under Juergen Schmidhuber, Istituto Dalle Molle di Studi sull'Intelligenza Artificiale (IDSIA) Former undergraduate researcher under Joshua Tenenbaum and Antonio Torralba, MIT

Antonia Creswell (Deep Mind)

Antonia Creswell is a Senior Research Scientist at DeepMind in the Cognition team. Her work focuses on the learning and integration of object representations in dynamic models. She completed her PhD on representation learning at Imperial College London in the department of Bioengineering.

Eliza Kosoy (UC Berkeley)

Working with Alison Gopnik (and a few others at MIT and Berkeley) on the intersections of ai and child development

Aravind Rajeswaran (University of Washington)
Sjoerd van Steenkiste (Dalle Molle Institute for Artificial Intelligence Research (IDSIA))

More from the Same Authors