Timezone: »
Ensemble-based debiasing methods have been shown effective in mitigating the reliance of classifiers on specific dataset bias, by exploiting the output of a bias-only model to adjust the learning target. In this paper, we focus on the bias-only model in these ensemble-based methods, which plays an important role but has not gained much attention in the existing literature. Theoretically, we prove that the debiasing performance can be damaged by inaccurate uncertainty estimations of the bias-only model. Empirically, we show that existing bias-only models fall short in producing accurate uncertainty estimations. Motivated by these findings, we propose to conduct calibration on the bias-only model, thus achieving a three-stage ensemble-based debiasing framework, including bias modeling, model calibrating, and debiasing. Experimental results on NLI and fact verification tasks show that our proposed three-stage debiasing framework consistently outperforms the traditional two-stage one in out-of-distribution accuracy.
Author Information
Ruibin Xiong (Baidu Inc.)
Yimeng Chen (Academy of Mathematics and Systems Science, Chinese Academy of Sciences)
Liang Pang (Institute of Computing Technology, Chinese Academy of Sciences)
Xueqi Cheng (ICT)
Zhi-Ming Ma
Yanyan Lan (Tsinghua University, Tsinghua University)
More from the Same Authors
-
2021 : Multi-modal Self-supervised Pre-training for Large-scale Genome Data »
Shentong Mo · Xi Fu · Chenyang Hong · Yizhen Chen · Yuxuan Zheng · Xiangru Tang · Yanyan Lan · Zhiqiang Shen · Eric Xing -
2022 Spotlight: Lightning Talks 4A-3 »
Zhihan Gao · Yabin Wang · Xingyu Qu · Luziwei Leng · Mingqing Xiao · Bohan Wang · Yu Shen · Zhiwu Huang · Xingjian Shi · Qi Meng · Yupeng Lu · Diyang Li · Qingyan Meng · Kaiwei Che · Yang Li · Hao Wang · Huishuai Zhang · Zongpeng Zhang · Kaixuan Zhang · Xiaopeng Hong · Xiaohan Zhao · Di He · Jianguo Zhang · Yaofeng Tu · Bin Gu · Yi Zhu · Ruoyu Sun · Yuyang (Bernie) Wang · Zhouchen Lin · Qinghu Meng · Wei Chen · Wentao Zhang · Bin CUI · Jie Cheng · Zhi-Ming Ma · Mu Li · Qinghai Guo · Dit-Yan Yeung · Tie-Yan Liu · Jianxing Liao -
2022 Spotlight: Does Momentum Change the Implicit Regularization on Separable Data? »
Bohan Wang · Qi Meng · Huishuai Zhang · Ruoyu Sun · Wei Chen · Zhi-Ming Ma · Tie-Yan Liu -
2022 Spotlight: Characterization of Excess Risk for Locally Strongly Convex Population Risk »
Mingyang Yi · Ruoyu Wang · Zhi-Ming Ma -
2022 Spotlight: When Does Group Invariant Learning Survive Spurious Correlations? »
Yimeng Chen · Ruibin Xiong · Zhi-Ming Ma · Yanyan Lan -
2022 Poster: When Does Group Invariant Learning Survive Spurious Correlations? »
Yimeng Chen · Ruibin Xiong · Zhi-Ming Ma · Yanyan Lan -
2022 Poster: Characterization of Excess Risk for Locally Strongly Convex Population Risk »
Mingyang Yi · Ruoyu Wang · Zhi-Ming Ma -
2022 Poster: Does Momentum Change the Implicit Regularization on Separable Data? »
Bohan Wang · Qi Meng · Huishuai Zhang · Ruoyu Sun · Wei Chen · Zhi-Ming Ma · Tie-Yan Liu -
2021 Poster: Self-Supervised GANs with Label Augmentation »
Liang Hou · Huawei Shen · Qi Cao · Xueqi Cheng -
2017 Poster: Finite sample analysis of the GTD Policy Evaluation Algorithms in Markov Setting »
Yue Wang · Wei Chen · Yuting Liu · Zhi-Ming Ma · Tie-Yan Liu -
2016 Poster: A Communication-Efficient Parallel Algorithm for Decision Tree »
Qi Meng · Guolin Ke · Taifeng Wang · Wei Chen · Qiwei Ye · Zhi-Ming Ma · Tie-Yan Liu -
2012 Poster: Statistical Consistency of Ranking Methods in A Rank-Differentiable Probability Space »
Yanyan Lan · Jiafeng Guo · Xueqi Cheng · Tie-Yan Liu -
2012 Spotlight: Statistical Consistency of Ranking Methods in A Rank-Differentiable Probability Space »
Yanyan Lan · Jiafeng Guo · Xueqi Cheng · Tie-Yan Liu -
2010 Poster: Two-Layer Generalization Analysis for Ranking Using Rademacher Average »
Wei Chen · Tie-Yan Liu · Zhi-Ming Ma -
2009 Poster: Ranking Measures and Loss Functions in Learning to Rank »
Wei Chen · Tie-Yan Liu · Yanyan Lan · Zhi-Ming Ma · Hang Li