Timezone: »

 
Oral
Bootstrap Your Own Latent - A New Approach to Self-Supervised Learning
Jean-Bastien Grill · Florian Strub · Florent Altché · Corentin Tallec · Pierre Richemond · Elena Buchatskaya · Carl Doersch · Bernardo Avila Pires · Daniel (Zhaohan) Guo · Mohammad Gheshlaghi Azar · Bilal Piot · koray kavukcuoglu · Remi Munos · Michal Valko

Thu Dec 10 06:15 AM -- 06:30 AM (PST) @ Orals & Spotlights: Unsupervised/Probabilistic
in Orals & Spotlights Track 27: Unsupervised/Probabilistic »

We introduce Bootstrap Your Own Latent (BYOL), a new approach to self-supervised image representation learning. BYOL relies on two neural networks, referred to as online and target networks, that interact and learn from each other. From an augmented view of an image, we train the online network to predict the target network representation of the same image under a different augmented view. At the same time, we update the target network with a slow-moving average of the online network. While state-of-the art methods intrinsically rely on negative pairs, BYOL achieves a new state of the art without them. BYOL reaches 74.3% top-1 classification accuracy on ImageNet using the standard linear evaluation protocol with a standard ResNet-50 architecture and 79.6% with a larger ResNet. We also show that BYOL performs on par or better than the current state of the art on both transfer and semi-supervised benchmarks.

Author Information

Jean-Bastien Grill (DeepMind)
Florian Strub (DeepMind)
Florent Altché (DeepMind)
Corentin Tallec (Deepmind)
Pierre Richemond (Imperial College)
Elena Buchatskaya (DeepMind)
Carl Doersch (DeepMind)
Bernardo Avila Pires (DeepMind)
Daniel (Zhaohan) Guo (DeepMind)
Mohammad Gheshlaghi Azar (DeepMind)
Bilal Piot (DeepMind)
koray kavukcuoglu (DeepMind)
Remi Munos (DeepMind)
Michal Valko (DeepMind)
Michal Valko

Michal is a machine learning scientist in DeepMind Paris, tenured researcher at Inria, and the lecturer of the master course Graphs in Machine Learning at l'ENS Paris-Saclay. Michal is primarily interested in designing algorithms that would require as little human supervision as possible. This means 1) reducing the “intelligence” that humans need to input into the system and 2) minimizing the data that humans need to spend inspecting, classifying, or “tuning” the algorithms. That is why he is working on methods and settings that are able to deal with minimal feedback, such as deep reinforcement learning, bandit algorithms, or self-supervised learning. Michal is actively working on represenation learning and building worlds models. He is also working on deep (reinforcement) learning algorithm that have some theoretical underpinning. He has also worked on sequential algorithms with structured decisions where exploiting the structure leads to provably faster learning. He received his Ph.D. in 2011 from the University of Pittsburgh under the supervision of Miloš Hauskrecht and after was a postdoc of Rémi Munos before taking a permanent position at Inria in 2012.

Related Events (a corresponding poster, oral, or spotlight)

More from the Same Authors