`

Timezone: »

 
Poster
Co-teaching: Robust training of deep neural networks with extremely noisy labels
Bo Han · Quanming Yao · Xingrui Yu · Gang Niu · Miao Xu · Weihua Hu · Ivor Tsang · Masashi Sugiyama

Tue Dec 04 07:45 AM -- 09:45 AM (PST) @ Room 517 AB #110
in Tue Poster Session A »

Deep learning with noisy labels is practically challenging, as the capacity of deep models is so high that they can totally memorize these noisy labels sooner or later during training. Nonetheless, recent studies on the memorization effects of deep neural networks show that they would first memorize training data of clean labels and then those of noisy labels. Therefore in this paper, we propose a new deep learning paradigm called ''Co-teaching'' for combating with noisy labels. Namely, we train two deep neural networks simultaneously, and let them teach each other given every mini-batch: firstly, each network feeds forward all data and selects some data of possibly clean labels; secondly, two networks communicate with each other what data in this mini-batch should be used for training; finally, each network back propagates the data selected by its peer network and updates itself. Empirical results on noisy versions of MNIST, CIFAR-10 and CIFAR-100 demonstrate that Co-teaching is much superior to the state-of-the-art methods in the robustness of trained deep models.

Keywords: Semi-Supervised Learning · Deep Learning
[ Paper] 

Author Information

Bo Han (RIKEN & UTS)
Quanming Yao (4Paradigm)
Xingrui Yu (University of Technology Sydney)

Ph.D Candidate at University of Technology Sydney.

Gang Niu (RIKEN)

Gang Niu is currently a research scientist (indefinite-term) at RIKEN Center for Advanced Intelligence Project. He received the PhD degree in computer science from Tokyo Institute of Technology in 2013. Before joining RIKEN as a research scientist, he was a senior software engineer at Baidu and then an assistant professor at the University of Tokyo. He has published more than 70 journal articles and conference papers, including 14 NeurIPS (1 oral and 3 spotlights), 28 ICML, and 2 ICLR (1 oral) papers. He has served as an area chair 14 times, including ICML 2019--2021, NeurIPS 2019--2021, and ICLR 2021--2022.

Miao Xu (RIKEN AIP)
Weihua Hu (The University of Tokyo)
Ivor Tsang (University of Technology, Sydney)
Masashi Sugiyama (RIKEN / University of Tokyo)

More from the Same Authors