Timezone: »

 
Poster
USB: A Unified Semi-supervised Learning Benchmark for Classification
Yidong Wang · Hao Chen · Yue Fan · Wang SUN · Ran Tao · Wenxin Hou · Renjie Wang · Linyi Yang · Zhi Zhou · Lan-Zhe Guo · Heli Qi · Zhen Wu · Yu-Feng Li · Satoshi Nakamura · Wei Ye · Marios Savvides · Bhiksha Raj · Takahiro Shinozaki · Bernt Schiele · Jindong Wang · Xing Xie · Yue Zhang

Wed Nov 30 02:00 PM -- 04:00 PM (PST) @ Hall J #1033
in Poster Session 4 »

Semi-supervised learning (SSL) improves model generalization by leveraging massive unlabeled data to augment limited labeled samples. However, currently, popular SSL evaluation protocols are often constrained to computer vision (CV) tasks. In addition, previous work typically trains deep neural networks from scratch, which is time-consuming and environmentally unfriendly. To address the above issues, we construct a Unified SSL Benchmark (USB) for classification by selecting 15 diverse, challenging, and comprehensive tasks from CV, natural language processing (NLP), and audio processing (Audio), on which we systematically evaluate the dominant SSL methods, and also open-source a modular and extensible codebase for fair evaluation of these SSL methods. We further provide the pre-trained versions of the state-of-the-art neural models for CV tasks to make the cost affordable for further tuning. USB enables the evaluation of a single SSL algorithm on more tasks from multiple domains but with less cost. Specifically, on a single NVIDIA V100, only 39 GPU days are required to evaluate FixMatch on 15 tasks in USB while 335 GPU days (279 GPU days on 4 CV datasets except for ImageNet) are needed on 5 CV tasks with TorchSSL.

Keywords: Semi-Supervised Learning
[ Paper]  [ Poster]  [ OpenReview] 

Author Information

Yidong Wang (Tokyo Institute of Technology)
Hao Chen (CMU, Carnegie Mellon University)
Yue Fan (Max-Planck-Institut for Informatics)
Wang SUN (Tsinghua University, Tsinghua University)
Ran Tao (Carnegie Mellon University)
Wenxin Hou (Microsoft)
Renjie Wang (Nanjing University)
Linyi Yang (Westlake University)
Zhi Zhou (Nanjing University)
Lan-Zhe Guo (Nanjing University)
Heli Qi (Nara Institute of Science and Technology, Japan)
Zhen Wu (Nanjing University)
Yu-Feng Li (Nanjing University)
Satoshi Nakamura (Nara Institute of Science and Technology, Japan)
Satoshi Nakamura

Satoshi Nakamura is a professor at the Graduate School of Science and Technology, Nara Institute of Science and Technology, Japan, and the Honorarprofessor of Karlsruhe Institute of Technology, Germany. He received a B.S. from the Kyoto Institute of Technology in 1981 and a Ph.D. from Kyoto University in 1992. He was a director of the ATR Spoken Language Communication Research Laboratories from 2000-2008 and vice president of ATR from 2007-2008, director general of the Keihanna Research Laboratories, National Institute of Information and Communications Technology, Japan in 2009-2010, and a project leader of the Tourism Information Analytics Team at Center for Advanced Intelligence Project AIP of RIKEN Institute in 2017-2021. He is currently the director of the Augmented Human Communication laboratory and a full professor of the Graduate School of Information Science at Nara Institute of Science and Technology. His interests include modeling and systems of speech-to-speech translation and speech recognition. He is one of the leaders of speech-to-speech translation research and has been serving in various worldwide speech-to-speech translation research projects, including C-STAR, IWSLT, and A-STAR. He received the LREC Antonio Zampolli Award in 2012. He also received the Commendation for Science and Technology by the Minister of Education, Science and Technology, and the Commendation for Science and Technology by the Minister of Internal Affairs and Communications. He was an elected board member of the International Speech Communication Association, ISCA in 2011-2018, an IEEE Signal Processing Magazine Editorial Board Member in 2012-2014, and an IEEE SPS Speech and Language Technical Committee Member in 2013-2015. He is IEEE Fellow, ISCA Fellow, IPSJ Fellow.

Wei Ye (Peking University)

Dr. Wei Ye is now an associate professor at National Engineering Research Center for Software Engineering, Peking University. In 2011, he obtained a doctorate degree from the School of Electronics Engineering and Computer Science, Peking University, working with Prof. Shikun Zhang. Wei Ye has a broad interest in real-world problems related to programming languages, natural languages, and knowledge graphs. More specifically, he is now conducting research into information extraction, natural language generation, and deep-learning-based program analysis.

Marios Savvides (Carnegie Mellon University)
Bhiksha Raj (Carnegie Mellon University)
Takahiro Shinozaki (Tokyo Institute of Technology)
Bernt Schiele (Max Planck Institute for Informatics)
Jindong Wang (Microsoft Research Asia)
Xing Xie (Microsoft Research Asia)
Yue Zhang (Westlake University)

More from the Same Authors