Timezone: »

 
Poster
Secure Out-of-Distribution Task Generalization with Energy-Based Models
Shengzhuang Chen · Long-Kai Huang · Jonathan Richard Schwarz · Yilun Du · Ying Wei

Thu Dec 14 03:00 PM -- 05:00 PM (PST) @ Great Hall & Hall B1+B2 #1124
in Poster Session 6 »

The success of meta-learning on out-of-distribution (OOD) tasks in the wild has proved to be hit-and-miss.To safeguard the generalization capability of the meta-learned prior knowledge to OOD tasks, in particularly safety-critical applications, necessitates detection of an OOD task followed by adaptation of the task towards the prior. Nonetheless, the reliability of estimated uncertainty on OOD tasks by existing Bayesian meta-learning methods is restricted by incomplete coverage of the feature distribution shift and insufficient expressiveness of the meta-learned prior. Besides, they struggle to adapt an OOD task, running parallel to the line of cross-domain task adaptation solutions which are vulnerable to overfitting.To this end, we build a single coherent framework that supports both detection and adaptation of OOD tasks, while remaining compatible with off-the-shelf meta-learning backbones. The proposed Energy-Based Meta-Learning (EBML) framework learns to characterize any arbitrary meta-training task distribution with the composition of two expressive neural-network-based energy functions. We deploy the sum of the two energy functions, being proportional to the joint distribution of a task, as a reliable score for detecting OOD tasks; during meta-testing, we adapt the OOD task to in-distribution tasks by energy minimization.Experiments on four regression and classification datasets demonstrate the effectiveness of our proposal.

Author Information

Shengzhuang Chen (City University of Hong Kong)
Long-Kai Huang (Tencent Technology（shenzhen）Company Limited)
Jonathan Richard Schwarz (DeepMind & UCL)
Yilun Du (Massachusetts Institute of Technology)
Ying Wei (Nanyang Technological University)

More from the Same Authors