Timezone: »

 
Poster
Pretraining Representations for Data-Efficient Reinforcement Learning
Max Schwarzer · Nitarshan Rajkumar · Michael Noukhovitch · Ankesh Anand · Laurent Charlin · R Devon Hjelm · Philip Bachman · Aaron Courville

Wed Dec 08 04:30 PM -- 06:00 PM (PST) @
in Poster Session 4 »

Data efficiency is a key challenge for deep reinforcement learning. We address this problem by using unlabeled data to pretrain an encoder which is then finetuned on a small amount of task-specific data. To encourage learning representations which capture diverse aspects of the underlying MDP, we employ a combination of latent dynamics modelling and unsupervised goal-conditioned RL. When limited to 100k steps of interaction on Atari games (equivalent to two hours of human experience), our approach significantly surpasses prior work combining offline representation pretraining with task-specific finetuning, and compares favourably with other pretraining methods that require orders of magnitude more data. Our approach shows particular promise when combined with larger models as well as more diverse, task-aligned observational data -- approaching human-level performance and data-efficiency on Atari in our best setting.

Keywords: Reinforcement Learning and Planning · Self-Supervised Learning

Author Information

Max Schwarzer (Mila, Université de Montréal)
Nitarshan Rajkumar (Mila, Université de Montréal)
Michael Noukhovitch (Mila (Université de Montréal))

Master's student at MILA supervised by Aaron Courville and co-supervised by Yoshua Bengio

Ankesh Anand (Mila, University of Montreal)
Laurent Charlin (MILA / U.Montreal)
R Devon Hjelm (Microsoft Research)
Philip Bachman (Microsoft Research)
Aaron Courville (U. Montreal)

More from the Same Authors