Out-of-Distribution Generalization: Shortcuts, Spuriousness, and Stability

Machine learning models often face challenges due to distribution shifts, leading to compromised performance during testing and limiting their use in high-stakes applications. For example, vision models have mistakenly relied on the height of shoulders in images to classify radiographs of COVID-19 patients, influenced by specific scanning techniques used during the pandemic's onset. Similarly, language models exhibit susceptibility to misleading syntactic patterns in natural language inference tasks like determining entailment, persisting as models grow in size. Addressing these issues requires characterizing relevant distribution shifts and establishing desired model behaviors under them.

This tutorial aims to provide a holistic perspective on distribution shifts due to spurious correlations and shortcut learning, as exemplified by the aforementioned instances. We situate existing research within a unified formal framework, discuss challenges in practical application of methods, and delineate the evolving landscape of research on spurious correlations in the era of foundation models. This tutorial serves as a compact and self-contained resource for students and researchers learning the topic, as well as practitioners seeking to deepen their understanding of the issues and of the tools to tackle them.

We will provide an overview of research trends, discuss available benchmarks, and propose best practices for future endeavors. The tutorial's final segment will focus on exploring spurious correlations in large models, culminating in a panel discussion on the subject.