Timezone: »

Latent World Models For Intrinsically Motivated Exploration
Aleksandr Ermolov · Nicu Sebe

Thu Dec 10 08:10 AM -- 08:20 AM (PST) @ Orals & Spotlights: Reinforcement Learning

In this work we consider partially observable environments with sparse rewards. We present a self-supervised representation learning method for image-based observations, which arranges embeddings respecting temporal distance of observations. This representation is empirically robust to stochasticity and suitable for novelty detection from the error of a predictive forward model. We consider episodic and life-long uncertainties to guide the exploration. We propose to estimate the missing information about the environment with the world model, which operates in the learned latent space. As a motivation of the method, we analyse the exploration problem in a tabular Partially Observable Labyrinth. We demonstrate the method on image-based hard exploration environments from the Atari benchmark and report significant improvement with respect to prior work. The source code of the method and all the experiments is available at https://github.com/htdt/lwm.

Author Information

Aleksandr Ermolov (University of Trento)
Nicu Sebe (University of Trento)

Related Events (a corresponding poster, oral, or spotlight)

More from the Same Authors