Timezone: »

 
Poster
Entropic Desired Dynamics for Intrinsic Control
Steven Hansen · Guillaume Desjardins · Kate Baumli · David Warde-Farley · Nicolas Heess · Simon Osindero · Volodymyr Mnih

Fri Dec 10 08:30 AM -- 10:00 AM (PST) @
in Poster Session 8 »

An agent might be said, informally, to have mastery of its environment when it has maximised the effective number of states it can reliably reach. In practice, this often means maximizing the number of latent codes that can be discriminated from future states under some short time horizon (e.g. \cite{eysenbach2018diversity}). By situating these latent codes in a globally consistent coordinate system, we show that agents can reliably reach more states in the long term while still optimizing a local objective. A simple instantiation of this idea, \textbf{E}ntropic \textbf{D}esired \textbf{D}ynamics for \textbf{I}ntrinsic \textbf{C}on\textbf{T}rol (EDDICT), assumes fixed additive latent dynamics, which results in tractable learning and an interpretable latent space. Compared to prior methods, EDDICT's globally consistent codes allow it to be far more exploratory, as demonstrated by improved state coverage and increased unsupervised performance on hard exploration games such as Montezuma's Revenge.

Keywords: Reinforcement Learning and Planning

Author Information

Steven Hansen (DeepMind)
Guillaume Desjardins (DeepMind)
Kate Baumli (DeepMind)
David Warde-Farley (DeepMind)
Nicolas Heess (Google DeepMind)
Simon Osindero (Flickr / Yahoo!)
Volodymyr Mnih (DeepMind)

More from the Same Authors