Timezone: »
Language is an interface to the outside world. In order for embodied agents to use it, language must be grounded in other, sensorimotor modalities. While there is an extended literature studying how machines can learn grounded language, the topic of how to learn spatio-temporal linguistic concepts is still largely uncharted. To make progress in this direction, we here introduce a novel spatio-temporal language grounding task where the goal is to learn the meaning of spatio-temporal descriptions of behavioral traces of an embodied agent. This is achieved by training a truth function that predicts if a description matches a given history of observations. The descriptions involve time-extended predicates in past and present tense as well as spatio-temporal references to objects in the scene. To study the role of architectural biases in this task, we train several models including multimodal Transformer architectures; the latter implement different attention computations between words and objects across space and time. We test models on two classes of generalization: 1) generalization to new sentences, 2) generalization to grammar primitives. We observe that maintaining object identity in the attention computation of our Transformers is instrumental to achieving good performance on generalization overall, and that summarizing object traces in a single token has little influence on performance. We then discuss how this opens new perspectives for language-guided autonomous embodied agents.
Author Information
Tristan Karch (Inria)
Laetitia Teodorescu (Inria)
Katja Hofmann (Microsoft Research)
Dr. Katja Hofmann is a Principal Researcher at the [Game Intelligence](http://aka.ms/gameintelligence/) group at [Microsoft Research Cambridge, UK](https://www.microsoft.com/en-us/research/lab/microsoft-research-cambridge/). There, she leads a research team that focuses on reinforcement learning with applications in modern video games. She and her team strongly believe that modern video games will drive a transformation of how we interact with AI technology. One of the projects developed by her team is [Project Malmo](https://www.microsoft.com/en-us/research/project/project-malmo/), which uses the popular game Minecraft as an experimentation platform for developing intelligent technology. Katja's long-term goal is to develop AI systems that learn to collaborate with people, to empower their users and help solve complex real-world problems. Before joining Microsoft Research, Katja completed her PhD in Computer Science as part of the [ILPS](https://ilps.science.uva.nl/) group at the [University of Amsterdam](https://www.uva.nl/en). She worked with Maarten de Rijke and Shimon Whiteson on interactive machine learning algorithms for search engines.
Clément Moulin-Frier (Universitat Pompeu Fabra)
I obtained my PhD in Engineering of Cognition, Interaction, Learning and Creation from Grenoble University in 2011. My main research interest is the modeling of social behavior formation in robotic agents, studying the role of cognitive, morphological, sensori-motor and environmental factors. In 2009 I have been a visiting scholar in Michael Arbib's lab at the University of Southern California in Los Angeles, USA. After a short contract at the College de France in Paris where I worked on probabilist optimal control for bidedal robots, I conducted my research between 2011 and 2014 in the Flowers group at Inria Bordeaux, in the field of developmental robotics. Since 2015 I have been working as a post-doctoral researcher in the Synthetic, Perceptive, Emotive and Cognitive Systems laboratory (SPECS), in particular on social robotics european projects such as WYSIWYD [1] and SocSMCs [2]. [1] http://wysiwyd.upf.edu [2] http://socsmcs.eu
Pierre-Yves Oudeyer (INRIA)
More from the Same Authors
-
2022 : Using Confounded Data in Offline RL »
Maxime Gasse · Damien GRASSET · Guillaume Gaudron · Pierre-Yves Oudeyer -
2022 : Contextual Squeeze-and-Excitation »
Massimiliano Patacchiola · John Bronskill · Aliaksandra Shysheya · Katja Hofmann · Sebastian Nowozin · Richard Turner -
2022 : Imitating Human Behaviour with Diffusion Models »
Tim Pearce · Tabish Rashid · Anssi Kanervisto · David Bignell · Mingfei Sun · Raluca Georgescu · Sergio Valcarcel Macua · Shan Zheng Tan · Ida Momennejad · Katja Hofmann · Sam Devlin -
2023 Workshop: Intrinsically Motivated Open-ended Learning (IMOL) Workshop »
Cédric Colas · Laetitia Teodorescu · Nadia Ady · Cansu Sancaktar · Junyi Chu -
2022 Workshop: LaReL: Language and Reinforcement Learning »
Laetitia Teodorescu · Laura Ruis · Tristan Karch · Cédric Colas · Paul Barde · Jelena Luketina · Athul Jacob · Pratyusha Sharma · Edward Grefenstette · Jacob Andreas · Marc-Alexandre Côté -
2022 Poster: EAGER: Asking and Answering Questions for Automatic Reward Shaping in Language-guided RL »
Thomas Carta · Pierre-Yves Oudeyer · Olivier Sigaud · Sylvain Lamprier -
2022 Poster: Uni[MASK]: Unified Inference in Sequential Decision Problems »
Micah Carroll · Orr Paradise · Jessy Lin · Raluca Georgescu · Mingfei Sun · David Bignell · Stephanie Milani · Katja Hofmann · Matthew Hausknecht · Anca Dragan · Sam Devlin -
2022 Poster: Contextual Squeeze-and-Excitation for Efficient Few-Shot Image Classification »
Massimiliano Patacchiola · John Bronskill · Aliaksandra Shysheya · Katja Hofmann · Sebastian Nowozin · Richard Turner -
2021 : Towards RL applications in video games and with human users »
Katja Hofmann -
2021 : Sculpting (human-like) AI systems by sculpting their (social) environments »
Pierre-Yves Oudeyer -
2021 : Methods:: Understanding Human-like Behavior in Video Game Navigation »
Evelyn Zuniga · Stephanie Milani · Katja Hofmann -
2021 : IGLU: Interactive Grounded Language Understanding in a Collaborative Environment + Q&A »
Julia Kiseleva · Ziming Li · Mohammad Aliannejadi · Maartje Anne ter Hoeve · Mikhail Burtsev · Alexey Skrynnik · Artem Zholus · Aleksandr Panov · Katja Hofmann · Kavya Srinet · arthur szlam · Michel Galley · Ahmed Awadallah -
2021 Poster: Memory Efficient Meta-Learning with Large Images »
John Bronskill · Daniela Massiceti · Massimiliano Patacchiola · Katja Hofmann · Sebastian Nowozin · Richard Turner -
2020 Workshop: Competition Track Saturday »
Hugo Jair Escalante · Katja Hofmann -
2020 : Panel discussion »
Pierre-Yves Oudeyer · Marc Bellemare · Peter Stone · Matt Botvinick · Susan Murphy · Anusha Nagabandi · Ashley Edwards · Karen Liu · Pieter Abbeel -
2020 : Invited talk: PierreYves Oudeyer "Machines that invent their own problems: Towards open-ended learning of skills" »
Pierre-Yves Oudeyer -
2020 Workshop: Competition Track Friday »
Hugo Jair Escalante · Katja Hofmann -
2020 : Opening - Competition Track Session »
Katja Hofmann · Hugo Jair Escalante -
2020 Poster: Hierarchically Organized Latent Modules for Exploratory Search in Morphogenetic Systems »
Mayalen Etcheverry · Clément Moulin-Frier · Pierre-Yves Oudeyer -
2020 Oral: Hierarchically Organized Latent Modules for Exploratory Search in Morphogenetic Systems »
Mayalen Etcheverry · Clément Moulin-Frier · Pierre-Yves Oudeyer -
2020 Poster: Language as a Cognitive Tool to Imagine Goals in Curiosity Driven Exploration »
Cédric Colas · Tristan Karch · Nicolas Lair · Jean-Michel Dussoux · Clément Moulin-Frier · Peter F Dominey · Pierre-Yves Oudeyer -
2020 : Discussion Panel: Hugo Larochelle, Finale Doshi-Velez, Devi Parikh, Marc Deisenroth, Julien Mairal, Katja Hofmann, Phillip Isola, and Michael Bowling »
Hugo Larochelle · Finale Doshi-Velez · Marc Deisenroth · Devi Parikh · Julien Mairal · Katja Hofmann · Phillip Isola · Michael Bowling -
2019 : Multi-Task Reinforcement Learning and Generalization »
Katja Hofmann -
2019 : The MineRL competition »
Misa Ogura · Joe Booth · Sophia Sun · Nicholay Topin · Brandon Houghton · William Guss · Stephanie Milani · Oriol Vinyals · Katja Hofmann · JIA KIM · Karolis Ramanauskas · Florian Laurent · Daichi Nishio · Anssi Kanervisto · Alexey Skrynnik · Artemij Amiranashvili · Christian Scheller · KAIXIN WANG · Yanick Schraner -
2019 Poster: Generalization in Reinforcement Learning with Selective Noise Injection and Information Bottleneck »
Maximilian Igl · Kamil Ciosek · Yingzhen Li · Sebastian Tschiatschek · Cheng Zhang · Sam Devlin · Katja Hofmann -
2019 Poster: Better Exploration with Optimistic Actor Critic »
Kamil Ciosek · Quan Vuong · Robert Loftin · Katja Hofmann -
2019 Spotlight: Better Exploration with Optimistic Actor Critic »
Kamil Ciosek · Quan Vuong · Robert Loftin · Katja Hofmann -
2019 Poster: Successor Uncertainties: Exploration and Uncertainty in Temporal Difference Learning »
David Janz · Jiri Hron · Przemysław Mazur · Katja Hofmann · José Miguel Hernández-Lobato · Sebastian Tschiatschek -
2019 Tutorial: Reinforcement Learning: Past, Present, and Future Perspectives »
Katja Hofmann -
2018 : How Players Speak to an Intelligent Game Character Using Natural Language Messages »
Katja Hofmann -
2017 : Panel: "How can we characterise the landscape of intelligent systems and locate human-like intelligence in it?" »
Josh Tenenbaum · Gary Marcus · Katja Hofmann -
2017 : Katja Hofmann: 'Video games and the road to collaborative AI' »
Katja Hofmann -
2016 Demonstration: Autonomous exploration, active learning and human guidance with open-source Poppy humanoid robot platform and Explauto library »
Sébastien Forestier · Yoan Mollard · Pierre-Yves Oudeyer -
2016 Demonstration: Project Malmo - Minecraft for AI Research »
Katja Hofmann · Matthew A Johnson · Fernando Diaz · Alekh Agarwal · Tim Hutton · David Bignell · Evelyne Viegas -
2012 Poster: Exploration in Model-based Reinforcement Learning by Empirically Estimating Learning Progress »
Manuel Lopes · Tobias Lang · Marc Toussaint · Pierre-Yves Oudeyer