Timezone: »
EnergyBased Models (EBM) capture dependencies between variables by associating a scalar energy to each configuration of the variables. Given a set of observed variables, an EBM inference consists in finding configurations of unobserved variables that minimize the energy. Training an EBM consists in designing a loss function whose minimization will shape the energy surface so that desired variable configurations have lower energies than undesired configurations. EBM approaches have been applied with considerable success to such problems as natural language processing, biological sequence analysis, computer vision (object detection and recognition), image segmentation, image restoration, unsupervised feature learning, and dimensionality reduction.
The first part of the tutorial will introduce the concepts of energybased inference, will discuss the relationships with nonprobabilistic forms of graphical models (unnormalized factor graphs), and will give the conditions that the loss function must satisfy so that its minimization will cause the model to produce good decisions. The second part will discuss the relative merits of EBM approaches and probabilistic approaches. EBMs provide more flexibility than probabilistic approaches in the design of the energy function because of the absence of normalization. More importantly, when training complex probabilistic models, one is often faced with the problem of evaluating (or approximating) intractable sums or integrals. EBMs trained with appropriate loss functions sidestep this problem altogether. The third part will present several popular learning models in the light of the EBM framework. In particular, discriminative learning methods for "structured" outputs will be discussed including: discriminative HMMs, Graph Transformer Networks, Conditional Random Fields, Maximum Margin Markov Networks, and related approaches. A simple interpretation will be given for several approximate maximum likelihood methods such as products of experts models, variational bound methods, and Hinton's Contrastive Divergence. Lastly, a number of applications to vision, NLP and bioinformatics will be discussed.
Author Information
Yann LeCun (Facebook AI Research and New York University)
Yann LeCun is VP & Chief AI Scientist at Facebook and Silver Professor at NYU affiliated with the Courant Institute of Mathematical Sciences & the Center for Data Science. He was the founding Director of Facebook AI Research and of the NYU Center for Data Science. He received an Engineering Diploma from ESIEE (Paris) and a PhD from Sorbonne Université. After a postdoc in Toronto he joined AT&T Bell Labs in 1988, and AT&T Labs in 1996 as Head of Image Processing Research. He joined NYU as a professor in 2003 and Facebook in 2013. His interests include AI machine learning, computer perception, robotics and computational neuroscience. He is the recipient of the 2018 ACM Turing Award (with Geoffrey Hinton and Yoshua Bengio) for "conceptual and engineering breakthroughs that have made deep neural networks a critical component of computing", a member of the National Academy of Sciences, the National Academy of Engineering and a Chevalier de la Légion d’Honneur.
More from the Same Authors

2020 : Panel Discussion & Closing »
Yejin Choi · Alexei Efros · Chelsea Finn · Kristen Grauman · Quoc V Le · Yann LeCun · Ruslan Salakhutdinov · Eric Xing 
2020 : QA: Yann LeCun »
Yann LeCun 
2020 : Invited Talk: Yann LeCun »
Yann LeCun 
2019 : TBD »
Yann LeCun 
2018 Poster: GLoMo: Unsupervised Learning of Transferable Relational Graphs »
Zhilin Yang · Jake Zhao · Bhuwan Dhingra · Kaiming He · William Cohen · Russ Salakhutdinov · Yann LeCun 
2017 : Panel Session »
Neil Lawrence · Finale DoshiVelez · Zoubin Ghahramani · Yann LeCun · Max Welling · Yee Whye Teh · Ole Winther 
2017 Tutorial: Geometric Deep Learning on Graphs and Manifolds »
Michael Bronstein · Joan Bruna · arthur szlam · Xavier Bresson · Yann LeCun 
2016 : Discussion panel »
Ian Goodfellow · Soumith Chintala · Arthur Gretton · Sebastian Nowozin · Aaron Courville · Yann LeCun · Emily Denton 
2016 : EnergyBased Adversarial Training and Video Prediction »
Yann LeCun 
2016 Symposium: Deep Learning Symposium »
Yoshua Bengio · Yann LeCun · Navdeep Jaitly · Roger Grosse 
2016 Poster: Disentangling factors of variation in deep representation using adversarial training »
Michael Mathieu · Junbo Jake Zhao · Junbo (Jake) Zhao · Aditya Ramesh · Pablo Sprechmann · Yann LeCun 
2016 Invited Talk: Predictive Learning »
Yann LeCun 
2015 : LargeScale Optimization for Deep Learning »
Yann LeCun 
2015 Poster: Learning to Linearize Under Uncertainty »
Ross Goroshin · Michael Mathieu · Yann LeCun 
2015 Poster: Characterlevel Convolutional Networks for Text Classification »
Xiang Zhang · Junbo (Jake) Zhao · Yann LeCun 
2015 Poster: Deep learning with Elastic Averaging SGD »
Sixin Zhang · Anna Choromanska · Yann LeCun 
2015 Spotlight: Deep learning with Elastic Averaging SGD »
Sixin Zhang · Anna Choromanska · Yann LeCun 
2015 Tutorial: Deep Learning »
Geoffrey E Hinton · Yoshua Bengio · Yann LeCun 
2014 Poster: Exploiting Linear Structure Within Convolutional Networks for Efficient Evaluation »
Emily Denton · Wojciech Zaremba · Joan Bruna · Yann LeCun · Rob Fergus 
2014 Poster: Joint Training of a Convolutional Network and a Graphical Model for Human Pose Estimation »
Jonathan J Tompson · Arjun Jain · Yann LeCun · Christoph Bregler 
2013 Workshop: Neural Information Processing Scaled for Bioacoustics : NIPS4B »
Hervé GLOTIN · Yann LeCun · Thierry Artières · Stephane Mallat · Ofer Tchernichovski · Xanadu Halkias 
2011 Workshop: Deep Learning and Unsupervised Feature Learning »
Yoshua Bengio · Adam Coates · Yann LeCun · Nicolas Le Roux · Andrew Y Ng 
2011 Poster: Structured sparse coding via lateral inhibition »
arthur d szlam · Karol Gregor · Yann LeCun 
2010 Workshop: Deep Learning and Unsupervised Feature Learning »
Honglak Lee · Marc'Aurelio Ranzato · Yoshua Bengio · Geoffrey E Hinton · Yann LeCun · Andrew Y Ng 
2010 Demonstration: NeuFlow: a dataflow processor for convolutional nets and other realtime algorithms »
Yann LeCun 
2010 Spotlight: Learning Convolutional Feature Hierarchies for Visual Recognition »
koray kavukcuoglu · Pierre Sermanet · YLan Boureau · Karol Gregor · Michael Mathieu · Yann LeCun 
2010 Poster: Learning Convolutional Feature Hierarchies for Visual Recognition »
koray kavukcuoglu · Pierre Sermanet · YLan Boureau · Karol Gregor · Michael Mathieu · Yann LeCun 
2010 Poster: Regularized estimation of image statistics by Score Matching »
Diederik Kingma · Yann LeCun 
2007 Poster: Sparse Feature Learning for Deep Belief Networks »
Marc'Aurelio Ranzato · YLan Boureau · Yann LeCun 
2006 Poster: Efficient Learning of Sparse Representations with an EnergyBased Model »
Marc'Aurelio Ranzato · Christopher Poultney · Sumit Chopra · Yann LeCun 
2006 Spotlight: Efficient Learning of Sparse Representations with an EnergyBased Model »
Marc'Aurelio Ranzato · Christopher Poultney · Sumit Chopra · Yann LeCun