Timezone: »

 
Poster
BYOL-Explore: Exploration by Bootstrapped Prediction
Zhaohan Guo · Shantanu Thakoor · Miruna Pislar · Bernardo Avila Pires · Florent Altché · Corentin Tallec · Alaa Saade · Daniele Calandriello · Jean-Bastien Grill · Yunhao Tang · Michal Valko · Remi Munos · Mohammad Gheshlaghi Azar · Bilal Piot

Thu Dec 01 02:00 PM -- 04:00 PM (PST) @ Hall J #911
in Poster Session 6 »

We present BYOL-Explore, a conceptually simple yet general approach for curiosity-driven exploration in visually complex environments. BYOL-Explore learns the world representation, the world dynamics and the exploration policy all-together by optimizing a single prediction loss in the latent space with no additional auxiliary objective. We show that BYOL-Explore is effective in DM-HARD-8, a challenging partially-observable continuous-action hard-exploration benchmark with visually rich 3-D environment. On this benchmark, we solve the majority of the tasks purely through augmenting the extrinsic reward with BYOL-Explore intrinsic reward, whereas prior work could only get off the ground with human demonstrations. As further evidence of the generality of BYOL-Explore, we show that it achieves superhuman performance on the ten hardest exploration games in Atari while having a much simpler design than other competitive agents.

Keywords: Representation Learning · Exploration · Deep Reinforcement Learning · Self-supervised learning
[ Paper]  [ Poster]  [ OpenReview] 

Author Information

Zhaohan Guo (DeepMind)
Shantanu Thakoor (Google)
Miruna Pislar (DeepMind)
Bernardo Avila Pires (DeepMind)
Florent Altché (DeepMind)
Corentin Tallec (INRIA)
Alaa Saade (DeepMind)
Daniele Calandriello (DeepMind)
Jean-Bastien Grill (DeepMind)
Yunhao Tang (Columbia University)

I am a PhD student at Columbia IEOR. My research interests are reinforcement learning and approximate inference.

Michal Valko (DeepMind)
Remi Munos (DeepMind)
Mohammad Gheshlaghi Azar (DeepMind)
Bilal Piot (DeepMind)

More from the Same Authors