Timezone: »

 
Choreographer: Learning and Adapting Skills in Imagination
Pietro Mazzaglia · Tim Verbelen · Bart Dhoedt · Alexandre Lacoste · Sai Rajeswar Mudumba
Event URL: https://openreview.net/forum?id=qe7WtUckT8i »

Unsupervised skill learning aims to learn a rich repertoire of behaviors without external supervision, providing artificial agents with the ability to control and influence the environment. However, without appropriate knowledge and exploration, skills may provide control only over a restricted area of the environment, limiting their applicability. Furthermore, it is unclear how to leverage the learned skill behaviors for adapting to downstream tasks in a data-efficient manner. We present Choreographer, a model-based agent that exploits its world model to learn and adapt skills in imagination. Our method decouples the exploration and skill learning processes, being able to discover skills in the latent state space of the model. During adaptation, the agent uses a meta-controller to evaluate and adapt the learned skills efficiently by deploying them in parallel in imagination. Choreographer is able to learn skills both from offline data, and by collecting data simultaneously with an exploration policy. The skills can be used to effectively adapt to downstream tasks, as we show in the URL benchmark, where we outperform previous approaches from both pixels and states inputs. The skills also explore the environment thoroughly, finding sparse rewards more frequently, as shown in goal-reaching tasks from the DMC Suite and Meta-World. Project website: https://doubleblind-repos.github.io/

Author Information

Pietro Mazzaglia (Ghent University)
Tim Verbelen (IDLab, Ghent University imec)
Bart Dhoedt (Ghent University)
Alexandre Lacoste (Service Now Research)
Sai Rajeswar Mudumba (ServiceNow)

More from the Same Authors

  • 2022 : Tensor networks for active inference with discrete observation spaces »
    Samuel T. Wauthier · Bram Vanhecke · Tim Verbelen · Bart Dhoedt
  • 2022 : Attention for Compositional Modularity »
    Oleksiy Ostapenko · Pau Rodriguez · Alexandre Lacoste · Laurent Charlin
  • 2022 : Choreographer: Learning and Adapting Skills in Imagination »
    Pietro Mazzaglia · Tim Verbelen · Bart Dhoedt · Alexandre Lacoste · Sai Rajeswar Mudumba
  • 2022 : Enforcing Object Permanence using Hierarchical Object-Centric Generative Models »
    Toon Van de Maele · Stefano Ferraro · Tim Verbelen · Bart Dhoedt
  • 2022 : Uncertainty in Neural Networks vs. Dermatologists for Skin Lesion Classification »
    Pieter Van Molle · Sofie Mylle · Tim Verbelen · Cedric De Boom · Bert Vankeirsbilck · Evelien Verhaeghe · Bart Dhoedt · Lieve Brochez
  • 2022 : Chunking Space and Time with Information Geometry »
    Tim Verbelen · Daria de Tinguy · Pietro Mazzaglia · Ozan Catal · Adam Safron
  • 2022 : A General-Purpose Neural Architecture for Geospatial Systems »
    Martin Weiss · Nasim Rahaman · Frederik Träuble · Francesco Locatello · Alexandre Lacoste · Yoshua Bengio · Erran Li Li · Chris Pal · Bernhard Schölkopf
  • 2022 : Chunking Space and Time with Information Geometry »
    Tim Verbelen · Daria de Tinguy · Pietro Mazzaglia · Ozan Catal · Adam Safron
  • 2021 Poster: Contrastive Active Inference »
    Pietro Mazzaglia · Tim Verbelen · Bart Dhoedt
  • 2019 : Lunch + Poster Session »
    Frederik Gerzer · Bill Yang Cai · Pieter-Jan Hoedt · Kelly Kochanski · Soo Kyung Kim · Yunsung Lee · Sunghyun Park · Sharon Zhou · Martin Gauch · Jonathan Wilson · Joyjit Chatterjee · Shamindra Shrotriya · Dimitri Papadimitriou · Christian Schön · Valentina Zantedeschi · Gabriella Baasch · Willem Waegeman · Gautier Cosne · Dara Farrell · Brendan Lucier · Letif Mones · Caleb Robinson · Tafara Chitsiga · Victor Kristof · Hari Prasanna Das · Yimeng Min · Alexandra Puchko · Alexandra Luccioni · Kyle Story · Jason Hickey · Yue Hu · Björn Lütjens · Zhecheng Wang · Renzhi Jing · Genevieve Flaspohler · Jingfan Wang · Saumya Sinha · Qinghu Tang · Armi Tiihonen · Ruben Glatt · Muge Komurcu · Jan Drgona · Juan Gomez-Romero · Ashish Kapoor · Dylan J Fitzpatrick · Alireza Rezvanifar · Adrian Albert · Olya (Olga) Irzak · Kara Lamb · Ankur Mahesh · Kiwan Maeng · Frederik Kratzert · Sorelle Friedler · Niccolo Dalmasso · Alex Robson · Lindiwe Malobola · Lucas Maystre · Yu-wen Lin · Surya Karthik Mukkavili · Brian Hutchinson · Alexandre Lacoste · Yanbing Wang · Zhengcheng Wang · Yinda Zhang · Victoria Preston · Jacob Pettit · Draguna Vrabie · Miguel Molina-Solana · Tonio Buonassisi · Andrew Annex · Tunai P Marques · Catalin Voss · Johannes Rausch · Max Evans
  • 2018 Poster: Towards Text Generation with Adversarially Learned Neural Outlines »
    Sandeep Subramanian · Sai Rajeswar Mudumba · Alessandro Sordoni · Adam Trischler · Aaron Courville · Chris Pal
  • 2017 Demonstration: A Deep Reinforcement Learning Chatbot »
    Iulian Vlad Serban · Chinnadhurai Sankar · Mathieu Germain · Saizheng Zhang · Zhouhan Lin · Sandeep Subramanian · Taesup Kim · Michael Pieper · Sarath Chandar · Nan Rosemary Ke · Sai Rajeswar Mudumba · Alexandre de Brébisson · Jose Sotelo · Dendi A Suhubdy · Vincent Michalski · Joelle Pineau · Yoshua Bengio