The ability to continuously learn and adapt itself to new tasks, without losing grasp of already acquired knowledge is a hallmark of biological learning systems, which current deep learning systems fall short of. In this work, we present a novel methodology for continual learning called MERLIN: Meta-Consolidation for Continual Learning.
We assume that weights of a neural network, for solving task, come from a meta-distribution. This meta-distribution is learned and consolidated incrementally. We operate in the challenging online continual learning setting, where a data point is seen by the model only once.
Our experiments with continual learning benchmarks of MNIST, CIFAR-10, CIFAR-100 and Mini-ImageNet datasets show consistent improvement over five baselines, including a recent state-of-the-art, corroborating the promise of MERLIN.
Joseph K J (Indian Institute of Technology Hyderabad)
Vineeth N Balasubramanian (Indian Institute of Technology, Hyderabad)
More from the Same Authors
2021 Poster: Adversarial Robustness without Adversarial Training: A Teacher-Guided Curriculum Learning Approach »
Anindya Sarkar · Anirban Sarkar · Sowrya Gali · Vineeth N Balasubramanian
2021 Poster: Can we have it all? On the Trade-off between Spatial and Adversarial Robustness of Neural Networks »
Sandesh Kamath · Amit Deshpande · Subrahmanyam Kambhampati Venkata · Vineeth N Balasubramanian