Timezone: »

 
Poster
Continual Deep Learning by Functional Regularisation of Memorable Past
Pingbo Pan · Siddharth Swaroop · Alexander Immer · Runa Eschenhagen · Richard Turner · Mohammad Emtiyaz Khan

Wed Dec 09 09:00 AM -- 11:00 AM (PST) @ Poster Session 3 #766
in Poster Session 4 »

Continually learning new skills is important for intelligent systems, yet standard deep learning methods suffer from catastrophic forgetting of the past. Recent works address this with weight regularisation. Functional regularisation, although computationally expensive, is expected to perform better, but rarely does so in practice. In this paper, we fix this issue by using a new functional-regularisation approach that utilises a few memorable past examples crucial to avoid forgetting. By using a Gaussian Process formulation of deep networks, our approach enables training in weight-space while identifying both the memorable past and a functional prior. Our method achieves state-of-the-art performance on standard benchmarks and opens a new direction for life-long learning where regularisation and memory-based methods are naturally combined.

Author Information

Pingbo Pan (University of Technology Sydney)
Siddharth Swaroop (University of Cambridge)
Alexander Immer (EPFL)
Runa Eschenhagen (University of Osnabrueck)
Richard Turner (University of Cambridge)
Mohammad Emtiyaz Khan (RIKEN, Tokyo)

Related Events (a corresponding poster, oral, or spotlight)

More from the Same Authors