Timezone: »

 
Learning compositional tasks from language instructions
Lajanugen Logeswaran · Wilka Carvalho · Honglak Lee

@
in Deep Reinforcement Learning »
Event URL: https://openreview.net/forum?id=CoMFsP9Vs-k »

Systematic compositionality - the ability to combine learned knowledge and skills to solve novel tasks -- is a key aspect of generalization in humans that allows us to understand and perform tasks described by novel language utterances. While progress has been made in supervised learning settings, no work has yet studied compositional generalization of a reinforcement learning agent following natural language instructions in an embodied environment. We develop a set of tasks in a photo-realistic simulated kitchen environment that allow us to study the degree to which a behavioral policy captures the systematicity in language by studying its zero-shot generalization performance on held out natural language instructions. We show that our agent which leverages a novel additive action-value decomposition in tandem with attention-based subgoal prediction is able to exploit composition in text instructions to generalize to unseen tasks.

Author Information

Lajanugen Logeswaran (University of Michigan)
Wilka Carvalho (University of Michigan)

I am a Masters student and NSF Graduate Research Fellow in the Computer Science Department at the University of Southern California. My primary research interest is the development of neuroscience- and cognitive science-informed artificial intelligence and machine learning models with brain-rivaling information-processing capabilities.

Honglak Lee (U. Michigan)

More from the Same Authors