Timezone: »
Meta-learning algorithms are widely used for few-shot learning. For example, image recognition systems that readily adapt to unseen classes after seeing only a few labeled examples. Despite their success, we show that modern meta-learning algorithms are extremely sensitive to the data used for adaptation, i.e. support data. In particular, we demonstrate the existence of (unaltered, in-distribution, natural) images that, when used for adaptation, yield accuracy as low as 4\% or as high as 95\% on standard few-shot image classification benchmarks. We explain our empirical findings in terms of class margins, which in turn suggests that robust and safe meta-learning requires larger margins than supervised learning.
Author Information
Mayank Agarwal (IBM Research AI, MIT-IBM Watson AI Lab)
Mikhail Yurochkin (IBM Research, MIT-IBM Watson AI Lab)
Yuekai Sun (University of Michigan)
More from the Same Authors
-
2021 : Measuring the sensitivity of Gaussian processes to kernel choice »
Will Stephenson · Soumya Ghosh · Tin Nguyen · Mikhail Yurochkin · Sameer Deshpande · Tamara Broderick -
2021 : COVID-19 India Dataset: Parsing Detailed COVID-19 Data in Daily Health Bulletins from States in India »
Mayank Agarwal · Tathagata Chakraborti · Sachin Grover -
2022 Poster: Domain Adaptation meets Individual Fairness. And they get along. »
Debarghya Mukherjee · Felix Petersen · Mikhail Yurochkin · Yuekai Sun -
2022 Poster: Calibrated Data-Dependent Constraints with Exact Satisfaction Guarantees »
Songkai Xue · Yuekai Sun · Mikhail Yurochkin -
2021 Poster: Does enforcing fairness mitigate biases caused by subpopulation shift? »
Subha Maity · Debarghya Mukherjee · Mikhail Yurochkin · Yuekai Sun -
2021 Poster: Post-processing for Individual Fairness »
Felix Petersen · Debarghya Mukherjee · Yuekai Sun · Mikhail Yurochkin -
2020 : NLC2CMD Competition Organizers: Metrics, Data, Tracks »
Mayank Agarwal -
2020 : NLC2CMD Competition Keynote: Tellina »
Victoria Lin · Mayank Agarwal · Tathagata Chakraborti -
2020 : NLC2CMD Competition Organizers: Introduction, Problem Description, CLAI »
Mayank Agarwal -
2020 : Spotlight Session 1 »
Augustus Odena · Maxwell Nye · Disha Shrivastava · Mayank Agarwal · Vincent J Hellendoorn · Charles Sutton -
2020 Poster: Continuous Regularized Wasserstein Barycenters »
Lingxiao Li · Aude Genevay · Mikhail Yurochkin · Justin Solomon -
2020 Demonstration: IBM Federated Learning Community Edition: An Interactive Demonstration »
Laura Wynter · Chaitanya Kumar · Pengqian Yu · Mikhail Yurochkin · Amogh Tarcar -
2019 Poster: Alleviating Label Switching with Optimal Transport »
Pierre Monteiller · Sebastian Claici · Edward Chien · Farzaneh Mirzazadeh · Justin Solomon · Mikhail Yurochkin -
2019 Poster: Hierarchical Optimal Transport for Document Representation »
Mikhail Yurochkin · Sebastian Claici · Edward Chien · Farzaneh Mirzazadeh · Justin Solomon -
2019 Demonstration: Project BB: Bringing AI to the Command Line »
Tathagata Chakraborti · Mayank Agarwal -
2019 Poster: Scalable inference of topic evolution via models for latent geometric structures »
Mikhail Yurochkin · Zhiwei Fan · Aritra Guha · Paraschos Koutris · XuanLong Nguyen -
2019 Poster: Statistical Model Aggregation via Parameter Matching »
Mikhail Yurochkin · Mayank Agarwal · Soumya Ghosh · Kristjan Greenewald · Nghia Hoang