Timezone: »

 
Poster
Learning values across many orders of magnitude
Hado van Hasselt · Arthur Guez · Arthur Guez · Matteo Hessel · Volodymyr Mnih · David Silver

Wed Dec 07 09:00 AM -- 12:30 PM (PST) @ Area 5+6+7+8 #83
in Wednesday Posters »

Most learning algorithms are not invariant to the scale of the signal that is being approximated. We propose to adaptively normalize the targets used in the learning updates. This is important in value-based reinforcement learning, where the magnitude of appropriate value approximations can change over time when we update the policy of behavior. Our main motivation is prior work on learning to play Atari games, where the rewards were clipped to a predetermined range. This clipping facilitates learning across many different games with a single learning algorithm, but a clipped reward function can result in qualitatively different behavior. Using adaptive normalization we can remove this domain-specific heuristic without diminishing overall performance.

Keywords: Deep Learning or Neural Networks · Reinforcement Learning Algorithms

Author Information

Hado van Hasselt (DeepMind)
Arthur Guez (DeepMind)
Arthur Guez (DeepMind)
Matteo Hessel (Google DeepMind)
Volodymyr Mnih (DeepMind)
David Silver (DeepMind)

More from the Same Authors