( events)   Timezone: »  
Mon Dec 13 09:00 AM -- 06:00 PM (PST)
Distribution shifts: connecting methods and applications (DistShift)
Shiori Sagawa · Pang Wei Koh · Fanny Yang · Hongseok Namkoong · Jiashi Feng · Kate Saenko · Percy Liang · Sarah Bird · Sergey Levine

Distribution shifts---where a model is deployed on a data distribution different from what it was trained on---pose significant robustness challenges in real-world ML applications. Such shifts are often unavoidable in the wild and have been shown to substantially degrade model performance in applications such as biomedicine, wildlife conservation, sustainable development, robotics, education, and criminal justice. For example, models can systematically fail when tested on patients from different hospitals or people from different demographics. Despite the ubiquity of distribution shifts in ML applications, work on these types of real-world shifts is currently underrepresented in the ML research community, with prior work generally focusing instead on synthetic shifts. However, recent work has shown that models that are robust to one kind of shift need not be robust to another, underscoring the importance and urgency of studying the types of distribution shifts that arise in real-world ML deployments. With this workshop, we aim to facilitate deeper exchanges between domain experts in various ML application areas and more methods-oriented researchers, and ground the development of methods for characterizing and mitigating distribution shifts in real-world application contexts.

Opening remarks (Talk)
Distribution Shifts in AI for Social Good (Invited talk)
Dataset Shifts: 8 Years of Going from Practice to Theory to Policy and Future Directions (Invited talk)
ML Model Debugging: A Data Perspective (Invited talk)
Discussion: Aleksander Mądry, Ernest Mwebaze, Suchi Saria (Panel)
Increasing Robustness to Distribution Shifts by Improving Design (Invited talk)
Statistical Testing under Distribution Shifts (Invited talk)
Discussion: Elizabeth Tipton, Jonas Peters (Panel)
Benchmarking Bayesian Deep Learning on Diabetic Retinopathy Detection Tasks (Spotlight)
A Closer Look at Distribution Shifts and Out-of-Distribution Generalization on Graphs (Spotlight)
On Adaptivity and Confounding in Contextual Bandit Experiments (Spotlight)
Is Importance Weighting Incompatible with Interpolating Classifiers? (Spotlight)
Poster session
Break / Lounge (Break)
Importance Weighting for Transfer Learning (Invited talk)
Robustness through the Lens of Invariance (Invited talk)
Discussion: Chelsea Finn, Masashi Sugiyama (Panel)
Panel: Future directions for tackling distribution shifts (Panel)
A Unified DRO View of Multi-class Loss Functions with top-N Consistency (Poster)
Test time Adaptation through Perturbation Robustness (Poster)
Are Vision Transformers Always More Robust Than Convolutional Neural Networks? (Poster)
Mix-MaxEnt: Improving Accuracy and Uncertainty Estimates of Deterministic Neural Networks (Poster)
Multi-Domain Ensembles for Domain Generalization (Poster)
Fourier-Based Augmentations for Improved Robustness and Uncertainty Calibration (Poster)
Learning Invariant Representations with Missing Data (Poster)
Using Distributionally Robust Optimization to improve robustness in cancer pathology (Poster)
Continual Density Ratio Estimation (Poster)
Maximum Mean Discrepancy for Generalization in the Presence of Distribution and Missingness Shift (Poster)
Effect of Model Size on Worst-group Generalization (Poster)
Correct-N-Contrast: A Contrastive Approach for Improving Robustness to Spurious Correlations (Poster)
Is Importance Weighting Incompatible with Interpolating Classifiers? (Poster)
PixMix: Dreamlike Pictures Comprehensively Improve Safety Measures (Poster)
Surprisingly Simple Semi-Supervised Domain Adaptation with Pretraining and Consistency (Poster)
The impact of domain shift on the calibration of fine-tuned models (Poster)
Handling Distribution Shift in Tire Design (Poster)
Augmented Self-Labeling for Source-Free Unsupervised Domain Adaptation (Poster)
Reliable Graph Neural Networks for Drug Discovery Under Distributional Shift (Poster)
Adversarial Training Blocks Generalization in Neural Policies (Poster)
Diurnal or Nocturnal? Federated Learning from Periodically Shifting Distributions (Poster)
PCA Subspaces Are Not Always Optimal for Bayesian Learning (Poster)
Calibrated Ensembles: A Simple Way to Mitigate ID-OOD Accuracy Tradeoffs (Poster)
Semi-Supervised Domain Generalization with Stochastic StyleMatch (Poster)
DANNTe: a case study of a turbo-machinery sensor virtualization under domain shift (Poster)
Model Zoo: A Growing Brain That Learns Continually (Poster)
Thinking Beyond Distributions in Testing Machine Learned Models (Poster)
Towards Robust and Adaptable Motion Forecasting: A Causal Representation Perspective (Poster)
Tackling Online One-Class Incremental Learning by Removing Negative Contrasts (Poster)
Improving Baselines in the Wild (Poster)
Re-labeling Domains Improves Multi-Domain Generalization (Poster)
Shift and Scale is Detrimental To Few-Shot Transfer (Poster)
Select, Label, and Mix: Learning Discriminative Invariant Feature Representations for Partial Domain Adaptation (Poster)
Investigating Shifts in GAN Output-Distributions (Poster)
Gradient-matching coresets for continual learning (Poster)
Just Mix Once: Mixing Samples with Implicit Group Distribution (Poster)
An Empirical Investigation of Model-to-Model Distribution Shifts in Trained Convolutional Filters (Poster)
On Adaptivity and Confounding in Contextual Bandit Experiments (Poster)
Towards Data-Free Domain Generalization (Poster)
Nonparametric Approach to Uncertainty Quantification for Deterministic Neural Networks (Poster)
Newer is not always better: Rethinking transferability metrics, their peculiarities, stability and performance (Poster)
Identifying the Instances Associated with Distribution Shifts using the Max-Sliced Bures Divergence (Poster)
An Empirical Study of Pre-trained Models on Out-of-distribution Generalization (Poster)
Exploring Covariate and Concept Shift for Out-of-Distribution Detection (Poster)
Understanding Post-hoc Adaptation for Improving Subgroup Robustness (Poster)
Robust fine-tuning of zero-shot models (Poster)
Probing Representation Forgetting in Continual Learning (Poster)
Catastrophic Failures of Neural Active Learning on Heteroskedastic Distributions (Poster)
Igeood: An Information Geometry Approach to Out-of-Distribution Detection (Poster)
Self-supervised Learning is More Robust to Dataset Imbalance (Poster)
Ensembles and Cocktails: Robust Finetuning for Natural Language Generation (Poster)
Understanding and Improving Robustness of VisionTransformers through patch-based NegativeAugmentation (Poster)
Mixture of Basis for Interpretable Continual Learning with Distribution Shifts (Poster)
Distribution Preserving Bayesian Coresets using Set Constraints (Poster)
Causal-based Time Series Domain Generalization for Vehicle Intention Prediction (Poster)
Leveraging Unlabeled Data to Predict Out-of-Distribution Performance (Poster)
Revisiting Visual Product for Compositional Zero-Shot Learning (Poster)
Quantifying and Alleviating Distribution Shifts in Foundation Models on Review Classification (Poster)
Randomly projecting out distribution shifts for improved robustness (Poster)
Avoiding Spurious Correlations: Bridging Theory and Practice (Poster)
On The Reliability Of Machine Learning Applications In Manufacturing Environments (Poster)
Distribution Shift in Airline Customer Behavior during COVID-19 (Poster)
A Closer Look at Distribution Shifts and Out-of-Distribution Generalization on Graphs (Poster)
Boosting worst-group accuracy without group annotations (Poster)
How Does Contrastive Pre-training Connect Disparate Domains? (Poster)
Benchmarking Robustness to Natural Distribution Shifts for Facial Analysis (Poster)
Domain-agnostic Test-time Adaptation by Prototypical Training with Auxiliary Data (Poster)
Jointly Learning from Decentralized (Federated) and Centralized Data to Mitigate Distribution Shift (Poster)
A fine-grained analysis of robustness to distribution shifts (Poster)
Test Time Robustification of Deep Models via Adaptation and Augmentation (Poster)
Internalized Biases in Fréchet Inception Distance (Poster)
Distributionally Robust Group Backwards Compatibility (Poster)
Unsupervised Attribute Alignment for Characterizing Distribution Shift (Poster)
Smooth Transfer Learning for Source-to-Target Generalization (Poster)
Benchmarking Bayesian Deep Learning on Diabetic Retinopathy Detection Tasks (Poster)
Optimal Representations for Covariate Shifts (Poster)
Con$^{2}$DA: Simplifying Semi-supervised Domain Adaptation by Learning Consistent and Contrastive Feature Representations (Poster)
Spectrally Adaptive Common Spatial Patterns (Poster)
A benchmark with decomposed distribution shifts for 360 monocular depth estimation (Poster)
Kernel Landmarks: An Empirical Statistical Approach to Detect Covariate Shift (Poster)
KitchenShift: Evaluating Zero-Shot Generalization of Imitation-Based Policy Learning Under Domain Shifts (Poster)
Exploiting Causal Chains for Domain Generalization (Poster)
BEDS-Bench: Behavior of EHR-models under Distributional Shift - A Benchmark (Poster)
Distribution Mismatch Correction for Improved Robustness in Deep Neural Networks (Poster)
Extending the WILDS Benchmark for Unsupervised Adaptation (Poster)