Timezone: »
Semi-supervised learning (SSL) provides an effective means of leveraging unlabeled data to improve a model’s performance. This domain has seen fast progress recently, at the cost of requiring more complex methods. In this paper we propose FixMatch, an algorithm that is a significant simplification of existing SSL methods. FixMatch first generates pseudo-labels using the model’s predictions on weakly-augmented unlabeled images. For a given image, the pseudo-label is only retained if the model produces a high-confidence prediction. The model is then trained to predict the pseudo-label when fed a strongly-augmented version of the same image. Despite its simplicity, we show that FixMatch achieves state-of-the-art performance across a variety of standard semi-supervised learning benchmarks, including 94.93% accuracy on CIFAR-10 with 250 labels and 88.61% accuracy with 40 – just 4 labels per class. We carry out an extensive ablation study to tease apart the experimental factors that are most important to FixMatch’s success. The code is available at https://github.com/google-research/fixmatch.
Author Information
Kihyuk Sohn (Google)
David Berthelot (Google Brain)
Nicholas Carlini (Google)
Zizhao Zhang (Google)
Han Zhang (Google)
Colin A Raffel (Google Brain)
My research focuses on machine learning techniques for sequential data. I am currently a resident at Google Brain. I recently completed a PhD in Electrical Engineering at Columbia University In LabROSA, supervised by Dan Ellis. My thesis focused on learning-based methods for comparing sequences. In 2010, I received a Master's in Music, Science and Technology from Stanford University's CCRMA, supervised by Julius O. Smith III. I did my undergrad at Oberlin College, where I majored in Mathematics.
Ekin Dogus Cubuk (Google Brain)
Alexey Kurakin (Google Brain)
Chun-Liang Li (Google)
More from the Same Authors
-
2020 : End-to-End Differentiability and Tensor Processing Unit Computing to Accelerate Materials’ Inverse Design »
HAN LIU · Yuhan Liu · Zhangji Zhao · Samuel Schoenholz · Ekin Dogus Cubuk · Mathieu Bauchy -
2021 Spotlight: Revisiting ResNets: Improved Training and Scaling Strategies »
Irwan Bello · William Fedus · Xianzhi Du · Ekin Dogus Cubuk · Aravind Srinivas · Tsung-Yi Lin · Jonathon Shlens · Barret Zoph -
2021 : Measuring Robustness to Natural Distribution Shifts in Image Classification »
Rohan Taori · Achal Dave · Vaishaal Shankar · Nicholas Carlini · Benjamin Recht · Ludwig Schmidt -
2022 : Part-Based Models Improve Adversarial Robustness »
Chawin Sitawarin · Kornrapat Pongmala · Yizheng Chen · Nicholas Carlini · David Wagner -
2022 : Invited talk: E. Doğuş Çubuk, "Scaling up material discovery via deep learning" »
Ekin Dogus Cubuk · Siddharth Mishra-Sharma -
2021 Poster: Controlling Neural Networks with Rule Representations »
Sungyong Seo · Sercan Arik · Jinsung Yoon · Xiang Zhang · Kihyuk Sohn · Tomas Pfister -
2021 Poster: Improved Transformer for High-Resolution GANs »
Long Zhao · Zizhao Zhang · Ting Chen · Dimitris Metaxas · Han Zhang -
2021 Poster: Object-aware Contrastive Learning for Debiased Scene Representation »
Sangwoo Mo · Hyunwoo Kang · Kihyuk Sohn · Chun-Liang Li · Jinwoo Shin -
2021 Poster: Revisiting ResNets: Improved Training and Scaling Strategies »
Irwan Bello · William Fedus · Xianzhi Du · Ekin Dogus Cubuk · Aravind Srinivas · Tsung-Yi Lin · Jonathon Shlens · Barret Zoph -
2020 : Contributed Talk #2: On the (Im)Possibility of Private Machine Learning through Instance Encoding »
Nicholas Carlini -
2020 Poster: Top-k Training of GANs: Improving GAN Performance by Throwing Away Bad Samples »
Samarth Sinha · Zhengli Zhao · Anirudh Goyal · Colin A Raffel · Augustus Odena -
2020 Poster: RandAugment: Practical Automated Data Augmentation with a Reduced Search Space »
Ekin Dogus Cubuk · Barret Zoph · Jonathon Shlens · Quoc V Le -
2020 Poster: JAX MD: A Framework for Differentiable Physics »
Samuel Schoenholz · Ekin Dogus Cubuk -
2020 Spotlight: JAX MD: A Framework for Differentiable Physics »
Samuel Schoenholz · Ekin Dogus Cubuk -
2020 Poster: On Completeness-aware Concept-Based Explanations in Deep Neural Networks »
Chih-Kuan Yeh · Been Kim · Sercan Arik · Chun-Liang Li · Tomas Pfister · Pradeep Ravikumar -
2020 Poster: Rethinking Pre-training and Self-training »
Barret Zoph · Golnaz Ghiasi · Tsung-Yi Lin · Yin Cui · Hanxiao Liu · Ekin Dogus Cubuk · Quoc V Le -
2020 Oral: Rethinking Pre-training and Self-training »
Barret Zoph · Golnaz Ghiasi · Tsung-Yi Lin · Yin Cui · Hanxiao Liu · Ekin Dogus Cubuk · Quoc V Le -
2020 Poster: Enabling certification of verification-agnostic networks via memory-efficient semidefinite programming »
Sumanth Dathathri · Krishnamurthy Dvijotham · Alexey Kurakin · Aditi Raghunathan · Jonathan Uesato · Rudy Bunel · Shreya Shankar · Jacob Steinhardt · Ian Goodfellow · Percy Liang · Pushmeet Kohli -
2020 Poster: Interpretable Sequence Learning for Covid-19 Forecasting »
Sercan Arik · Chun-Liang Li · Jinsung Yoon · Rajarishi Sinha · Arkady Epshteyn · Long Le · Vikas Menon · Shashank Singh · Leyou Zhang · Martin Nikoltchev · Yash Sonthalia · Hootan Nakhost · Elli Kanal · Tomas Pfister -
2020 Spotlight: Interpretable Sequence Learning for Covid-19 Forecasting »
Sercan Arik · Chun-Liang Li · Jinsung Yoon · Rajarishi Sinha · Arkady Epshteyn · Long Le · Vikas Menon · Shashank Singh · Leyou Zhang · Martin Nikoltchev · Yash Sonthalia · Hootan Nakhost · Elli Kanal · Tomas Pfister -
2019 Poster: A Fourier Perspective on Model Robustness in Computer Vision »
Dong Yin · Raphael Gontijo Lopes · Jonathon Shlens · Ekin Dogus Cubuk · Justin Gilmer -
2019 Poster: MixMatch: A Holistic Approach to Semi-Supervised Learning »
David Berthelot · Nicholas Carlini · Ian Goodfellow · Nicolas Papernot · Avital Oliver · Colin A Raffel -
2018 : Poster Session 1 »
Evan Casey · Colin A Raffel · Jonathan Simon · Juncheng Li · Robert Saunders · Petra Gemeinboeck · Eunsu Kang · Songwei Ge · Curtis Hawthorne · Anna Huang · Ting-Wei Su · Eric Chu · Memo Akten · Sonam Damani · Khyatti Gupta · Dilpreet Singh · Patrick Hutchings -
2018 : Adversarial Vision Challenge: Results of the Adversarial Vision Challenge »
Wieland Brendel · Jonas Rauber · Marcel Salathé · Alexey Kurakin · Nicolas Papernot · Sharada Mohanty · Matthias Bethge -
2018 Poster: Realistic Evaluation of Deep Semi-Supervised Learning Algorithms »
Avital Oliver · Augustus Odena · Colin A Raffel · Ekin Dogus Cubuk · Ian Goodfellow -
2018 Spotlight: Realistic Evaluation of Deep Semi-Supervised Learning Algorithms »
Avital Oliver · Augustus Odena · Colin A Raffel · Ekin Dogus Cubuk · Ian Goodfellow -
2018 Poster: Adversarial Examples that Fool both Computer Vision and Time-Limited Humans »
Gamaleldin Elsayed · Shreya Shankar · Brian Cheung · Nicolas Papernot · Alexey Kurakin · Ian Goodfellow · Jascha Sohl-Dickstein -
2016 Poster: Improved Deep Metric Learning with Multi-class N-pair Loss Objective »
Kihyuk Sohn -
2015 Poster: Learning Structured Output Representation using Deep Conditional Generative Models »
Kihyuk Sohn · Honglak Lee · Xinchen Yan -
2014 Poster: Improved Multimodal Deep Learning with Variation of Information »
Kihyuk Sohn · Wenling Shang · Honglak Lee -
2013 Workshop: Output Representation Learning »
Yuhong Guo · Dale Schuurmans · Richard Zemel · Samy Bengio · Yoshua Bengio · Li Deng · Dan Roth · Kilian Q Weinberger · Jason Weston · Kihyuk Sohn · Florent Perronnin · Gabriel Synnaeve · Pablo R Strasser · julien audiffren · Carlo Ciliberto · Dan Goldwasser