( events)   Timezone: »  
Mon Dec 13 09:00 AM -- 06:00 PM (PST)
Distribution shifts: connecting methods and applications (DistShift)
Shiori Sagawa · Pang Wei Koh · Fanny Yang · Hongseok Namkoong · Jiashi Feng · Kate Saenko · Percy Liang · Sarah Bird · Sergey Levine

Workshop Home Page

Distribution shifts---where a model is deployed on a data distribution different from what it was trained on---pose significant robustness challenges in real-world ML applications. Such shifts are often unavoidable in the wild and have been shown to substantially degrade model performance in applications such as biomedicine, wildlife conservation, sustainable development, robotics, education, and criminal justice. For example, models can systematically fail when tested on patients from different hospitals or people from different demographics. Despite the ubiquity of distribution shifts in ML applications, work on these types of real-world shifts is currently underrepresented in the ML research community, with prior work generally focusing instead on synthetic shifts. However, recent work has shown that models that are robust to one kind of shift need not be robust to another, underscoring the importance and urgency of studying the types of distribution shifts that arise in real-world ML deployments. With this workshop, we aim to facilitate deeper exchanges between domain experts in various ML application areas and more methods-oriented researchers, and ground the development of methods for characterizing and mitigating distribution shifts in real-world application contexts.

Opening remarks (Talk)
Invited talk: Aleksander Mądry (Invited talk)
Invited talk: Suchi Saria (Invited talk)
Invited talk: Ernest Mwebaze (Invited talk)
Discussion: Aleksander, Ernest, Suchi (Panel)
Invited talk: Elizabeth Tipton (Invited talk)
Invited talk: Jonas Peters (Invited talk)
Discussion: Elizabeth, Jonas (Panel)
Benchmarking Bayesian Deep Learning on Diabetic Retinopathy Detection Tasks (Spotlight)
A Closer Look at Distribution Shifts and Out-of-Distribution Generalization on Graphs (Spotlight)
On Adaptivity and Confounding in Contextual Bandit Experiments (Spotlight)
Is Importance Weighting Incompatible with Interpolating Classifiers? (Spotlight)
Invited talk: Chelsea Finn (Invited talk)
Invited talk: Masashi Sugiyama (Invited talk)
Discussion: Chelsea, Masashi (Panel)
Panel: Andrew Beck, Jamie Morgenstern, Judy Hoffman, Tatsunori Hashimoto (Panel)
Catastrophic Failures of Neural Active Learning on Heteroskedastic Distributions (Poster)
Correct-N-Contrast: A Contrastive Approach for Improving Robustness to Spurious Correlations (Poster)
DANNTe: a case study of a turbo-machinery sensor virtualization under domain shift (Poster)
Understanding Post-hoc Adaptation for Improving Subgroup Robustness (Poster)
Probing Representation Forgetting in Continual Learning (Poster)
Adversarial Training Blocks Generalization in Neural Policies (Poster)
KitchenShift: Evaluating Zero-Shot Generalization of Imitation-Based Policy Learning Under Domain Shifts (Poster)
Diurnal or Nocturnal? Federated Learning from Periodically Shifting Distributions (Poster)
Quantifying Model Predictive Uncertainty with Perturbation Theory (Poster)
Just Mix Once: Mixing Samples with Implicit Group Distribution (Poster)
Distribution Mismatch Correction for Improved Robustness in Deep Neural Networks (Poster)
Augmented Self-Labeling for Source-Free Unsupervised Domain Adaptation (Poster)
Avoiding Spurious Correlations: Bridging Theory and Practice (Poster)
Are Vision Transformers Always More Robust Than Convolutional Neural Networks? (Poster)
Identifying the Instances Associated with Distribution Shifts using the Max-Sliced Bures Divergence (Poster)
A Closer Look at Distribution Shifts and Out-of-Distribution Generalization on Graphs (Poster)
Self-supervised Learning is More Robust to Dataset Imbalance (Poster)
Towards Data-Free Domain Generalization (Poster)
Calibrated Ensembles: A Simple Way to Mitigate ID-OOD Accuracy Tradeoffs (Poster)
Towards Robust and Adaptable Motion Forecasting: A Causal Representation Perspective (Poster)
Learning Invariant Representations with Missing Data (Poster)
Investigating Shifts in GAN Output-Distributions (Poster)
Newer is not always better: Rethinking transferability metrics, their peculiarities, stability and performance (Poster)
Mix-MaxEnt: Improving Accuracy and Uncertainty Estimates of Deterministic Neural Networks (Poster)
Understanding and Improving Robustness of VisionTransformers through patch-based NegativeAugmentation (Poster)
PCA Subspaces Are Not Always Optimal for Bayesian Learning (Poster)
Randomly projecting out distribution shifts for improved robustness (Poster)
Test Time Robustification of Deep Models via Adaptation and Augmentation (Poster)
How Does Contrastive Pre-training Connect Disparate Domains? (Poster)
Leveraging Unlabeled Data to Predict Out-of-Distribution Performance (Poster)
The impact of domain shift on the calibration of fine-tuned models (Poster)
Internalized Biases in Fréchet Inception Distance (Poster)
Towards Reliable Machine Learning Applications in Dynamic Manufacturing Environments (Poster)
A fine-grained analysis of robustness to distribution shifts (Poster)
Exploiting Causal Chains for Domain Generalization (Poster)
Improving Baselines in the Wild (Poster)
Smooth Transfer Learning for Source-to-Target Generalization (Poster)
On Adaptivity and Confounding in Contextual Bandit Experiments (Poster)
Using Distributionally Robust Optimization to improve robustness in cancer pathology (Poster)
Benchmarking Robustness to Natural Distribution Shifts for Facial Analysis (Poster)
Revisiting Visual Product for Compositional Zero-Shot Learning (Poster)
Kernel Landmarks: An Empirical Statistical Approach to Detect Covariate Shift (Poster)
Maximum Mean Discrepancy for Generalization in the Presence of Distribution and Missingness Shift (Poster)
Thinking Beyond Distributions in Testing Machine Learned Models (Poster)
BEDS-Bench: Behavior of EHR-models under Distributional Shift - A Benchmark (Poster)
Handling Distribution Shift in Tire Design (Poster)
Quantifying and Alleviating Distribution Shifts in Foundation Models on Review Classification (Poster)
Boosting worst-group accuracy without group annotations (Poster)
Reliable Graph Neural Networks for Drug Discovery Under Distributional Shift (Poster)
Domain-agnostic Test-time Adaptation by Prototypical Training with Auxiliary Data (Poster)
An Empirical Investigation of Model-to-Model Distribution Shifts in Trained Convolutional Filters (Poster)
Extending the WILDS Benchmark for Unsupervised Adaptation (Poster)
Jointly Learning from Decentralized (Federated) and Centralized Data to Mitigate Distribution Shift (Poster)
A Unified DRO View of Multi-class Loss Functions with top-N Consistency (Poster)
Distribution Shift in Airline Customer Behavior during COVID-19 (Poster)
Multi-Domain Ensembles for Domain Generalization (Poster)
Effect of Model Size on Worst-group Generalization (Poster)
An Empirical Study of Pre-trained Models on Out-of-distribution Generalization (Poster)
PixMix: Dreamlike Pictures Comprehensively Improve Safety Measures (Poster)
A benchmark with decomposed distribution shifts for 360 monocular depth estimation (Poster)
Test time Adaptation through Perturbation Robustness (Poster)
Is Importance Weighting Incompatible with Interpolating Classifiers? (Poster)
Select, Label, and Mix: Learning Discriminative Invariant Feature Representations for Partial Domain Adaptation (Poster)
Distribution Preserving Bayesian Coresets using Set Constraints (Poster)
Benchmarking Bayesian Deep Learning on Diabetic Retinopathy Detection Tasks (Poster)
Surprisingly Simple Semi-Supervised Domain Adaptation with Pretraining and Consistency (Poster)
Mixture of Basis for Interpretable Continual Learning with Distribution Shifts (Poster)
Shift and Scale is Detrimental To Few-Shot Transfer (Poster)
Causal-based Time Series Domain Generalization for Vehicle Intention Prediction (Poster)
Model Zoo: A Growing Brain That Learns Continually (Poster)
Spectrally Adaptive Common Spatial Patterns (Poster)
Ensembles and Cocktails: Robust Finetuning for Natural Language Generation (Poster)
Con$^{2}$DA: Simplifying Semi-supervised Domain Adaptation by Learning Consistent and Contrastive Feature Representations (Poster)
Optimal Representations for Covariate Shifts (Poster)
Continual Density Ratio Estimation (Poster)
Igeood: An Information Geometry Approach to Out-of-Distribution Detection (Poster)
Robust fine-tuning of zero-shot models (Poster)
Unsupervised Attribute Alignment for Characterizing Distribution Shift (Poster)
Nonparametric Approach to Uncertainty Quantification for Deterministic Neural Networks (Poster)
Semi-Supervised Domain Generalization with Stochastic StyleMatch (Poster)
Gradient-matching coresets for continual learning (Poster)
Distributionally Robust Group Backwards Compatibility (Poster)
Tackling Online One-Class Incremental Learning by Removing Negative Contrasts (Poster)
Exploring Covariate and Concept Shift for Out-of-Distribution Detection (Poster)
Re-labeling Domains Improves Multi-Domain Generalization (Poster)
Fourier-Based Augmentations for Improved Robustness and Uncertainty Calibration (Poster)