Timezone: »

 
Poster
Value Prediction Network
Junhyuk Oh · Satinder Singh · Honglak Lee

Mon Dec 04 06:30 PM -- 10:30 PM (PST) @ Pacific Ballroom #206
in Posters Mon »

This paper proposes a novel deep reinforcement learning (RL) architecture, called Value Prediction Network (VPN), which integrates model-free and model-based RL methods into a single neural network. In contrast to typical model-based RL methods, VPN learns a dynamics model whose abstract states are trained to make option-conditional predictions of future values (discounted sum of rewards) rather than of future observations. Our experimental results show that VPN has several advantages over both model-free and model-based baselines in a stochastic environment where careful planning is required but building an accurate observation-prediction model is difficult. Furthermore, VPN outperforms Deep Q-Network (DQN) on several Atari games even with short-lookahead planning, demonstrating its potential as a new way of learning a good state representation.

Keywords: Reinforcement Learning · Model-Based RL · Planning
[ Code]  [ Paper] 

Author Information

Junhyuk Oh (DeepMind)
Satinder Singh (University of Michigan)
Honglak Lee (Google / U. Michigan)

More from the Same Authors