Timezone: »

 
Poster
Binary Classification with Confidence Difference
Wei Wang · Lei Feng · Yuchen Jiang · Gang Niu · Min-Ling Zhang · Masashi Sugiyama

Tue Dec 12 08:45 AM -- 10:45 AM (PST) @ Great Hall & Hall B1+B2 #1025
in Poster Session 1 »

Recently, learning with soft labels has been shown to achieve better performance than learning with hard labels in terms of model generalization, calibration, and robustness. However, collecting pointwise labeling confidence for all training examples can be challenging and time-consuming in real-world scenarios. This paper delves into a novel weakly supervised binary classification problem called confidence-difference (ConfDiff) classification. Instead of pointwise labeling confidence, we are given only unlabeled data pairs with confidence difference that specifies the difference in the probabilities of being positive. We propose a risk-consistent approach to tackle this problem and show that the estimation error bound achieves the optimal convergence rate. We also introduce a risk correction approach to mitigate overfitting problems, whose consistency and convergence rate are also proven. Extensive experiments on benchmark data sets and a real-world recommender system data set validate the effectiveness of our proposed approaches in exploiting the supervision information of the confidence difference.

Author Information

Wei Wang (The University of Tokyo / RIKEN)
Lei Feng (Nanyang Technological University)
Yuchen Jiang (Alibaba Group)
Gang Niu (RIKEN)
Gang Niu

Gang Niu is currently an indefinite-term senior research scientist at RIKEN Center for Advanced Intelligence Project.

Min-Ling Zhang (Southeast University)
Masashi Sugiyama (RIKEN / University of Tokyo)

More from the Same Authors