This is the public, feature-limited version of the conference webpage. After Registration and login please visit the full version.

Look-ahead Meta Learning for Continual Learning

Gunshi Gupta, Karmesh Yadav, Liam Paull

Oral presentation: Orals & Spotlights Track 16: Continual/Meta/Misc Learning
on 2020-12-09T06:15:00-08:00 - 2020-12-09T06:30:00-08:00
Poster Session 4 (more posters)
on 2020-12-09T09:00:00-08:00 - 2020-12-09T11:00:00-08:00
Abstract: The continual learning problem involves training models with limited capacity to perform well on a set of an unknown number of sequentially arriving tasks. While meta-learning shows great potential for reducing interference between old and new tasks, the current training procedures tend to be either slow or offline, and sensitive to many hyper-parameters. In this work, we propose Look-ahead MAML (La-MAML), a fast optimisation-based meta-learning algorithm for online-continual learning, aided by a small episodic memory. By incorporating the modulation of per-parameter learning rates in our meta-learning update, our approach also allows us to draw connections to and exploit prior work on hypergradients and meta-descent. This provides a more flexible and efficient way to mitigate catastrophic forgetting compared to conventional prior-based methods. La-MAML achieves performance superior to other replay-based, prior-based and meta-learning based approaches for continual learning on real-world visual classification benchmarks.

Preview Video and Chat

To see video, interact with the author and ask questions please use registration and login.