Timezone: »

 
It's COMPASlicated: The Messy Relationship between RAI Datasets and Algorithmic Fairness Benchmarks
Michelle Bao · Angela Zhou · Samantha Zottola · Brian Brubach · Sarah Desmarais · Aaron Horowitz · Kristian Lum · Suresh Venkatasubramanian

@ Virtual
in Dataset and Benchmark Poster Session 3 »
Event URL: https://openreview.net/forum?id=qeM58whnpXM »

Risk assessment instrument (RAI) datasets, particularly ProPublica’s COMPAS dataset, are commonly used in algorithmic fairness papers due to benchmarking practices of comparing algorithms on datasets used in prior work. In many cases, this data is used as a benchmark to demonstrate good performance without ac-counting for the complexities of criminal justice (CJ) processes. However, we show that pretrial RAI datasets can contain numerous measurement biases and errors, and due to disparities in discretion and deployment, algorithmic fairness applied to RAI datasets is limited in making claims about real-world outcomes.These reasons make the datasets a poor fit for benchmarking under assumptions of ground truth and real-world impact. Furthermore, conventional practices of simply replicating previous data experiments may implicitly inherit or edify normative positions without explicitly interrogating value-laden assumptions. Without con-text of how interdisciplinary fields have engaged in CJ research and context of how RAIs operate upstream and downstream, algorithmic fairness practices are misaligned for meaningful contribution in the context of CJ, and would benefit from transparent engagement with normative considerations and values related to fairness, justice, and equality. These factors prompt questions about whether benchmarks for intrinsically socio-technical systems like the CJ system can exist in a beneficial and ethical way.

[ Slides]  [ Chat]  [ OpenReview] 

Author Information

Michelle Bao (Stanford University)

:D

Angela Zhou (Cornell University)
Samantha Zottola (Policy Research Associates)
Brian Brubach (Wellesley College)
Sarah Desmarais (Policy Research Associates, Inc.)
Aaron Horowitz (American Civil Liberties Union)
Kristian Lum (Twitter)
Suresh Venkatasubramanian (University of Utah)

More from the Same Authors