Timezone: »
Energy-Based Models (EBM) capture dependencies between variables by associating a scalar energy to each configuration of the variables. Given a set of observed variables, an EBM inference consists in finding configurations of unobserved variables that minimize the energy. Training an EBM consists in designing a loss function whose minimization will shape the energy surface so that desired variable configurations have lower energies than undesired configurations. EBM approaches have been applied with considerable success to such problems as natural language processing, biological sequence analysis, computer vision (object detection and recognition), image segmentation, image restoration, unsupervised feature learning, and dimensionality reduction.
The first part of the tutorial will introduce the concepts of energy-based inference, will discuss the relationships with non-probabilistic forms of graphical models (un-normalized factor graphs), and will give the conditions that the loss function must satisfy so that its minimization will cause the model to produce good decisions. The second part will discuss the relative merits of EBM approaches and probabilistic approaches. EBMs provide more flexibility than probabilistic approaches in the design of the energy function because of the absence of normalization. More importantly, when training complex probabilistic models, one is often faced with the problem of evaluating (or approximating) intractable sums or integrals. EBMs trained with appropriate loss functions sidestep this problem altogether. The third part will present several popular learning models in the light of the EBM framework. In particular, discriminative learning methods for "structured" outputs will be discussed including: discriminative HMMs, Graph Transformer Networks, Conditional Random Fields, Maximum Margin Markov Networks, and related approaches. A simple interpretation will be given for several approximate maximum likelihood methods such as products of experts models, variational bound methods, and Hinton's Contrastive Divergence. Lastly, a number of applications to vision, NLP and bio-informatics will be discussed.
Author Information
Yann LeCun (Facebook)
Yann LeCun is VP & Chief AI Scientist at Meta and Silver Professor at NYU affiliated with the Courant Institute of Mathematical Sciences & the Center for Data Science. He was the founding Director of FAIR (Meta's AI Research group) and of the NYU Center for Data Science. He received an Engineering Diploma from ESIEE (Paris) and a PhD from Sorbonne Université. After a postdoc in Toronto he joined AT&T Bell Labs in 1988, and AT&T Labs in 1996 as Head of Image Processing Research. He joined NYU as a professor in 2003 and Facebook in 2013. His interests include AI machine learning, computer perception, robotics and computational neuroscience. He is the recipient of the 2018 ACM Turing Award (with Geoffrey Hinton and Yoshua Bengio) for "conceptual and engineering breakthroughs that have made deep neural networks a critical component of computing", a member of the National Academy of Sciences, the National Academy of Engineering and a Chevalier de la Légion d’Honneur.
More from the Same Authors
-
2021 : Deep generative models create new and diverse protein structures »
Zeming Lin · Tom Sercu · yann lecun · Alex Rives -
2023 Poster: Self-Supervised Learning with Lie Symmetries for Partial Differential Equations »
Grégoire Mialon · Quentin Garrido · Hannah Lawrence · Danyal Rehman · Bobak Kiani · Yann LeCun -
2023 Poster: Reverse Engineering Self-Supervised Learning »
Ido Ben-Shaul · Ravid Shwartz-Ziv · Tomer Galanti · Shai Dekel · Yann LeCun -
2023 Poster: An Information Theory Perspective on Variance-Invariance-Covariance Regularization »
Ravid Shwartz-Ziv · Randall Balestriero · Kenji Kawaguchi · Tim G. J. Rudner · Yann LeCun -
2022 Poster: The Effects of Regularization and Data Augmentation are Class Dependent »
Randall Balestriero · Leon Bottou · Yann LeCun -
2022 Poster: VICRegL: Self-Supervised Learning of Local Visual Features »
Adrien Bardes · Jean Ponce · Yann LeCun -
2022 Poster: Coarse-to-Fine Vision-Language Pre-training with Fusion in the Backbone »
Zi-Yi Dou · Aishwarya Kamath · Zhe Gan · Pengchuan Zhang · Jianfeng Wang · Linjie Li · Zicheng Liu · Ce Liu · Yann LeCun · Nanyun Peng · Jianfeng Gao · Lijuan Wang -
2022 Poster: Pre-Train Your Loss: Easy Bayesian Transfer Learning with Informative Priors »
Ravid Shwartz-Ziv · Micah Goldblum · Hossein Souri · Sanyam Kapoor · Chen Zhu · Yann LeCun · Andrew Wilson -
2022 Poster: A Data-Augmentation Is Worth A Thousand Samples: Analytical Moments And Sampling-Free Training »
Randall Balestriero · Ishan Misra · Yann LeCun -
2022 Poster: projUNN: efficient method for training deep networks with unitary matrices »
Bobak Kiani · Randall Balestriero · Yann LeCun · Seth Lloyd -
2022 Poster: Contrastive and Non-Contrastive Self-Supervised Learning Recover Global and Local Spectral Embedding Methods »
Randall Balestriero · Yann LeCun -
2021 : Deep generative models create new and diverse protein structures »
Zeming Lin · Tom Sercu · yann lecun · Alex Rives -
2020 : Panel Discussion & Closing »
Yejin Choi · Alexei Efros · Chelsea Finn · Kristen Grauman · Quoc V Le · Yann LeCun · Ruslan Salakhutdinov · Eric Xing -
2020 : QA: Yann LeCun »
Yann LeCun -
2020 : Invited Talk: Yann LeCun »
Yann LeCun -
2020 Poster: Implicit Rank-Minimizing Autoencoder »
Li Jing · Jure Zbontar · yann lecun -
2019 : TBD »
Yann LeCun -
2018 Poster: GLoMo: Unsupervised Learning of Transferable Relational Graphs »
Zhilin Yang · Jake Zhao · Bhuwan Dhingra · Kaiming He · William Cohen · Russ Salakhutdinov · Yann LeCun -
2017 : Panel Session »
Neil Lawrence · Finale Doshi-Velez · Zoubin Ghahramani · Yann LeCun · Max Welling · Yee Whye Teh · Ole Winther -
2017 Tutorial: Geometric Deep Learning on Graphs and Manifolds »
Michael Bronstein · Joan Bruna · arthur szlam · Xavier Bresson · Yann LeCun -
2016 : Discussion panel »
Ian Goodfellow · Soumith Chintala · Arthur Gretton · Sebastian Nowozin · Aaron Courville · Yann LeCun · Emily Denton -
2016 : Energy-Based Adversarial Training and Video Prediction »
Yann LeCun -
2016 Symposium: Deep Learning Symposium »
Yoshua Bengio · Yann LeCun · Navdeep Jaitly · Roger Grosse -
2016 Poster: Disentangling factors of variation in deep representation using adversarial training »
Michael Mathieu · Junbo Jake Zhao · Junbo (Jake) Zhao · Aditya Ramesh · Pablo Sprechmann · Yann LeCun -
2016 Invited Talk: Predictive Learning »
Yann LeCun -
2015 : Large-Scale Optimization for Deep Learning »
Yann LeCun -
2015 Poster: Learning to Linearize Under Uncertainty »
Ross Goroshin · Michael Mathieu · Yann LeCun -
2015 Poster: Character-level Convolutional Networks for Text Classification »
Xiang Zhang · Junbo (Jake) Zhao · Yann LeCun -
2015 Poster: Deep learning with Elastic Averaging SGD »
Sixin Zhang · Anna Choromanska · Yann LeCun -
2015 Spotlight: Deep learning with Elastic Averaging SGD »
Sixin Zhang · Anna Choromanska · Yann LeCun -
2015 Tutorial: Deep Learning »
Geoffrey E Hinton · Yoshua Bengio · Yann LeCun -
2014 Poster: Exploiting Linear Structure Within Convolutional Networks for Efficient Evaluation »
Emily Denton · Wojciech Zaremba · Joan Bruna · Yann LeCun · Rob Fergus -
2014 Poster: Joint Training of a Convolutional Network and a Graphical Model for Human Pose Estimation »
Jonathan J Tompson · Arjun Jain · Yann LeCun · Christoph Bregler -
2013 Workshop: Neural Information Processing Scaled for Bioacoustics : NIPS4B »
Hervé GLOTIN · Yann LeCun · Thierry Artières · Stephane Mallat · Ofer Tchernichovski · Xanadu Halkias -
2011 Workshop: Deep Learning and Unsupervised Feature Learning »
Yoshua Bengio · Adam Coates · Yann LeCun · Nicolas Le Roux · Andrew Y Ng -
2011 Poster: Structured sparse coding via lateral inhibition »
arthur d szlam · Karol Gregor · Yann LeCun -
2010 Workshop: Deep Learning and Unsupervised Feature Learning »
Honglak Lee · Marc'Aurelio Ranzato · Yoshua Bengio · Geoffrey E Hinton · Yann LeCun · Andrew Y Ng -
2010 Demonstration: NeuFlow: a dataflow processor for convolutional nets and other real-time algorithms »
Yann LeCun -
2010 Spotlight: Learning Convolutional Feature Hierarchies for Visual Recognition »
koray kavukcuoglu · Pierre Sermanet · Y-Lan Boureau · Karol Gregor · Michael Mathieu · Yann LeCun -
2010 Poster: Learning Convolutional Feature Hierarchies for Visual Recognition »
koray kavukcuoglu · Pierre Sermanet · Y-Lan Boureau · Karol Gregor · Michael Mathieu · Yann LeCun -
2010 Poster: Regularized estimation of image statistics by Score Matching »
Diederik Kingma · Yann LeCun -
2007 Poster: Sparse Feature Learning for Deep Belief Networks »
Marc'Aurelio Ranzato · Y-Lan Boureau · Yann LeCun -
2006 Poster: Efficient Learning of Sparse Representations with an Energy-Based Model »
Marc'Aurelio Ranzato · Christopher Poultney · Sumit Chopra · Yann LeCun -
2006 Spotlight: Efficient Learning of Sparse Representations with an Energy-Based Model »
Marc'Aurelio Ranzato · Christopher Poultney · Sumit Chopra · Yann LeCun