Timezone: »

 
Poster
Are My Deep Learning Systems Fair? An Empirical Study of Fixed-Seed Training
Shangshu Qian · Viet Hung Pham · Thibaud Lutellier · Zeou Hu · Jungwon Kim · Lin Tan · Yaoliang Yu · Jiahao Chen · Sameena Shah

Thu Dec 09 08:30 AM -- 10:00 AM (PST) @
in Poster Session 6 »

Deep learning (DL) systems have been gaining popularity in critical tasks such as credit evaluation and crime prediction. Such systems demand fairness. Recent work shows that DL software implementations introduce variance: identical DL training runs (i.e., identical network, data, configuration, software, and hardware) with a fixed seed produce different models. Such variance could make DL models and networks violate fairness compliance laws, resulting in negative social impact. In this paper, we conduct the first empirical study to quantify the impact of software implementation on the fairness and its variance of DL systems. Our study of 22 mitigation techniques and five baselines reveals up to 12.6% fairness variance across identical training runs with identical seeds. In addition, most debiasing algorithms have a negative impact on the model such as reducing model accuracy, increasing fairness variance, or increasing accuracy variance. Our literature survey shows that while fairness is gaining popularity in artificial intelligence (AI) related conferences, only 34.4% of the papers use multiple identical training runs to evaluate their approach, raising concerns about their results’ validity. We call for better fairness evaluation and testing protocols to improve fairness and fairness variance of DL systems as well as DL research validity and reproducibility at large.

Keywords: Deep Learning · Fairness

Author Information

Shangshu Qian (Purdue University)
Viet Hung Pham (University of Waterloo)
Thibaud Lutellier (University of Waterloo)
Zeou Hu (University of Waterloo)
Jungwon Kim (Purdue University)
Lin Tan (Purdue University)
Yaoliang Yu (Carnegie Mellon University)
Jiahao Chen (JPMorgan AI Research)
Sameena Shah (J.P. Morgan Chase)

More from the Same Authors