Timezone: »

 
Poster
Experience Replay for Continual Learning
David Rolnick · Arun Ahuja · Jonathan Richard Schwarz · Timothy Lillicrap · Gregory Wayne

Tue Dec 10 05:30 PM -- 07:30 PM (PST) @ East Exhibition Hall B + C #202
in Reinforcement Learning and Planning -- Reinforcement Learning »

Interacting with a complex world involves continual learning, in which tasks and data distributions change over time. A continual learning system should demonstrate both plasticity (acquisition of new knowledge) and stability (preservation of old knowledge). Catastrophic forgetting is the failure of stability, in which new experience overwrites previous experience. In the brain, replay of past experience is widely believed to reduce forgetting, yet it has been largely overlooked as a solution to forgetting in deep reinforcement learning. Here, we introduce CLEAR, a replay-based method that greatly reduces catastrophic forgetting in multi-task reinforcement learning. CLEAR leverages off-policy learning and behavioral cloning from replay to enhance stability, as well as on-policy learning to preserve plasticity. We show that CLEAR performs better than state-of-the-art deep learning techniques for mitigating forgetting, despite being significantly less complicated and not requiring any knowledge of the individual tasks being learned.

Keywords: Memory · Plasticity and Adaptation · Neuroscience and Cognitive Science
[ Paper]  [ Slides] 

Author Information

David Rolnick (UPenn)
Arun Ahuja (DeepMind)
Jonathan Richard Schwarz (DeepMind & Gatsby Unit, UCL)
Timothy Lillicrap (DeepMind & UCL)
Gregory Wayne (Google DeepMind)

More from the Same Authors