Timezone: »

 
A Nested Bi-level Optimization Framework for Robust Few Shot Learning
Krishnateja Killamsetty · Changbin Li · Chen Zhao · Rishabh Iyer · Feng Chen

@
in 5th Workshop on Meta-Learning »
Event URL: https://openreview.net/forum?id=OtokjoNoFu5 »

Model-Agnostic Meta-Learning (MAML), a popular gradient-based meta-learning framework, assumes that the contribution of each task or instance to the meta-learner is equal. Hence, it fails to address the domain shift between base and novel classes in few-shot learning. In this work, we propose a novel robust meta-learning algorithm, NESTEDMAML, which learns to assign weights to training tasks or instances. We consider weights as hyper-parameters and iteratively optimize them using a small set of validation tasks set in a nested bi-level optimization approach (in contrast to the standard bi-level optimization in MAML). We then apply NESTEDMAML in the meta-training stage, which involves (1) several tasks sampled from a distribution different from the meta-test task distribution, or (2) some data samples with noisy labels. Extensive experiments on synthetic and real-world datasets demonstrate that NESTEDMAML efficiently mitigates the effects of "unwanted" tasks or instances, leading to significant improvement over the state-of-the-art robust meta-learning methods.

Author Information

Krishnateja Killamsetty (University of Texas, Dallas)
Changbin Li (University of Texas, Dallas)

I am a Ph.D. Candidate with seven years of experience in machine learning, deep learning, computer vision, medical image processing, recommender system, NLP, with particular interests in low-resource learning (meta-learning, few-shot learning, semi/self-supervised learning, transfer learning); secure learning (fairness, uncertainty estimation); Efficient learning (dynamic neural networks), etc. I am looking for an AI/ML internship opportunity that will enable me to continue to grow in my role as an AI/ML Researcher.

Chen Zhao (University of Texas at Dallas)
Rishabh Iyer (University of Texas at Dallas)

Rishabh Iyer is currently an assistant professor at the University of Texas at Dallas. Prior to this, he was a Research Scientist at Microsoft. During his time at Microsoft, several of his algorithms and innovations have been shipped in Microsoft products including Microsoft Office and Bing ads. He finished his PostDoc and Ph.D. from the University of Washington, Seattle. His work has received the best paper awards at ICML and NIPS in 2013. He also won the Microsoft Ph.D. fellowship and Facebook Ph.D. Fellowship, along with the Yang Outstanding Doctoral Student Award from the University of Washington. He is interested in several aspects of machine learning including discrete and convex optimization, submodular optimization, deep learning, video/image summarization, diversified active learning, and data selection, online learning, robust learning, and inference, and structured prediction.

Feng Chen (UT Dallas)

More from the Same Authors