Timezone: »
Predictions about people, such as their expected educational achievement or their credit risk, can be performative and shape the outcome that they are designed to predict. Understanding the causal effect of predictions on the eventual outcomes is crucial for foreseeing the implications of future predictive models and selecting which models to deploy. However, this causal estimation task poses unique challenges: model predictions are usually deterministic functions of input features and highly correlated with outcomes, which can make the causal effects of predictions on outcomes impossible to disentangle from the direct effect of the covariates. We study this problem through the lens of causal identifiability. Despite the hardness of this problem in full generality, we highlight three natural scenarios where the causal effect of predictions can be identified from observational data: randomization in predictions, overparameterization of the predictive model deployed during data collection, and discrete prediction outputs. Empirically we show that given our identifiability conditions hold, standard variants of supervised learning that predict from predictions by treating the prediction as an input feature can find transferable functional relationships that allow for conclusions about newly deployed predictive models. These positive results fundamentally rely on model predictions being recorded during data collection, bringing forward the importance of rethinking standard data collection practices to enable progress towards a better understanding of social outcomes and performative feedback loops.
Author Information
Celestine Mendler-Dünner (Max Planck Institute for Intelligent Systems)
Frances Ding (University of California Berkeley)
Yixin Wang (University of Michigan)
More from the Same Authors
-
2021 Spotlight: Learning Equilibria in Matching Markets from Bandit Feedback »
Meena Jagadeesan · Alexander Wei · Yixin Wang · Michael Jordan · Jacob Steinhardt -
2021 : Alternative Microfoundations for Strategic Classification »
Meena Jagadeesan · Celestine Mendler-Dünner · Moritz Hardt -
2021 : Desiderata for Representation Learning: A Causal Perspective »
Yixin Wang · Michael Jordan -
2021 : Alternative Microfoundations for Strategic Classification »
Meena Jagadeesan · Celestine Mendler-Dünner · Moritz Hardt -
2022 : A Bayesian Causal Inference Approach for Assessing Fairness in Clinical Decision-Making »
Linying Zhang · Lauren Richter · Yixin Wang · Anna Ostropolets · Noemie Elhadad · David Blei · George Hripcsak -
2022 : Causal Inference out of Control: Identifying the Steerability of Consumption »
Gary Cheng · Moritz Hardt · Celestine Mendler-Dünner -
2022 : Valid Inference after Causal Discovery »
Paula Gradu · Tijana Zrnic · Yixin Wang · Michael Jordan -
2022 : Interventional Causal Representation Learning »
Kartik Ahuja · Yixin Wang · Divyat Mahajan · Yoshua Bengio -
2022 : Interventional Causal Representation Learning »
Kartik Ahuja · Yixin Wang · Divyat Mahajan · Yoshua Bengio -
2022 : Interventional Causal Representation Learning »
Kartik Ahuja · Yixin Wang · Divyat Mahajan · Yoshua Bengio -
2022 Workshop: Learning Meaningful Representations of Life »
Elizabeth Wood · Adji Bousso Dieng · Aleksandrina Goeva · Alex X Lu · Anshul Kundaje · Chang Liu · Debora Marks · Ed Boyden · Eli N Weinstein · Lorin Crawford · Mor Nitzan · Rebecca Boiarsky · Romain Lopez · Tamara Broderick · Ray Jones · Wouter Boomsma · Yixin Wang · Stephen Ra -
2022 : Panel »
Meena Jagadeesan · Avrim Blum · Jon Kleinberg · Celestine Mendler-Dünner · Jennifer Wortman Vaughan · Chara Podimata -
2022 : Causal Inference out of Control: Identifying the Steerability of Consumption »
Gary Cheng · Moritz Hardt · Celestine Mendler-Dünner -
2022 : Dynamic Survival Transformers for Causal Inference with Electronic Health Records »
Prayag Chatha · Yixin Wang · Zhenke Wu · Jeffrey Regier -
2022 : Dynamic Survival Transformers for Causal Inference with Electronic Health Records »
Prayag Chatha · Yixin Wang · Zhenke Wu · Jeffrey Regier -
2022 Poster: Performative Power »
Moritz Hardt · Meena Jagadeesan · Celestine Mendler-Dünner -
2022 Poster: Empirical Gateaux Derivatives for Causal Inference »
Michael Jordan · Yixin Wang · Angela Zhou -
2021 : Invited Talk 6 Q&A »
Yixin Wang -
2021 : Statistical and Computational Tradeoffs in Variational Bayes »
Yixin Wang -
2021 Workshop: Learning and Decision-Making with Strategic Feedback (StratML) »
Yahav Bechavod · Hoda Heidari · Eric Mazumdar · Celestine Mendler-Dünner · Tijana Zrnic -
2021 Workshop: Learning Meaningful Representations of Life (LMRL) »
Elizabeth Wood · Adji Bousso Dieng · Aleksandrina Goeva · Anshul Kundaje · Barbara Engelhardt · Chang Liu · David Van Valen · Debora Marks · Edward Boyden · Eli N Weinstein · Lorin Crawford · Mor Nitzan · Romain Lopez · Tamara Broderick · Ray Jones · Wouter Boomsma · Yixin Wang -
2021 Oral: Retiring Adult: New Datasets for Fair Machine Learning »
Frances Ding · Moritz Hardt · John Miller · Ludwig Schmidt -
2021 Poster: Grounding Representation Similarity Through Statistical Testing »
Frances Ding · Jean-Stanislas Denain · Jacob Steinhardt -
2021 Poster: Retiring Adult: New Datasets for Fair Machine Learning »
Frances Ding · Moritz Hardt · John Miller · Ludwig Schmidt -
2021 Poster: Test-time Collective Prediction »
Celestine Mendler-Dünner · Wenshuo Guo · Stephen Bates · Michael Jordan -
2021 Poster: Posterior Collapse and Latent Variable Non-identifiability »
Yixin Wang · David Blei · John Cunningham -
2021 Poster: Learning Equilibria in Matching Markets from Bandit Feedback »
Meena Jagadeesan · Alexander Wei · Yixin Wang · Michael Jordan · Jacob Steinhardt -
2020 Workshop: Learning Meaningful Representations of Life (LMRL.org) »
Elizabeth Wood · Debora Marks · Ray Jones · Adji Bousso Dieng · Alan Aspuru-Guzik · Anshul Kundaje · Barbara Engelhardt · Chang Liu · Edward Boyden · Kresten Lindorff-Larsen · Mor Nitzan · Smita Krishnaswamy · Wouter Boomsma · Yixin Wang · David Van Valen · Orr Ashenberg -
2020 Poster: Point process models for sequence detection in high-dimensional neural spike trains »
Alex Williams · Anthony Degleris · Yixin Wang · Scott Linderman -
2020 Oral: Point process models for sequence detection in high-dimensional neural spike trains »
Alex Williams · Anthony Degleris · Yixin Wang · Scott Linderman -
2020 Poster: Stochastic Optimization for Performative Prediction »
Celestine Mendler-Dünner · Juan Perdomo · Tijana Zrnic · Moritz Hardt -
2019 Poster: Variational Bayes under Model Misspecification »
Yixin Wang · David Blei -
2019 Poster: Using Embeddings to Correct for Unobserved Confounding in Networks »
Victor Veitch · Yixin Wang · David Blei -
2019 Poster: SySCD: A System-Aware Parallel Coordinate Descent Algorithm »
Nikolas Ioannou · Celestine Mendler-Dünner · Thomas Parnell -
2019 Spotlight: SySCD: A System-Aware Parallel Coordinate Descent Algorithm »
Nikolas Ioannou · Celestine Mendler-Dünner · Thomas Parnell -
2018 Poster: Snap ML: A Hierarchical Framework for Machine Learning »
Celestine Dünner · Thomas Parnell · Dimitrios Sarigiannis · Nikolas Ioannou · Andreea Anghel · Gummadi Ravi · Madhusudanan Kandasamy · Haralampos Pozidis -
2017 Poster: Efficient Use of Limited-Memory Accelerators for Linear Learning on Heterogeneous Systems »
Celestine Dünner · Thomas Parnell · Martin Jaggi