Timezone: »
Boltzmann machines are able to learn highly complex, multimodal, structured and multiscale real-world data distributions. Parameters of the model are usually learned by minimizing the Kullback-Leibler (KL) divergence from training samples to the learned model. We propose in this work a novel approach for Boltzmann machine training which assumes that a meaningful metric between observations is given. This metric can be represented by the Wasserstein distance between distributions, for which we derive a gradient with respect to the model parameters. Minimization of this new objective leads to generative models with different statistical properties. We demonstrate their practical potential on data completion and denoising, for which the metric between observations plays a crucial role.
Author Information
Grégoire Montavon (TU Berlin)
Klaus-Robert Müller (TU Berlin)
Marco Cuturi (Apple)
Marco Cuturi is a research scientist at Apple, in Paris. He received his Ph.D. in 11/2005 from the Ecole des Mines de Paris in applied mathematics. Before that he graduated from National School of Statistics (ENSAE) with a master degree (MVA) from ENS Cachan. He worked as a post-doctoral researcher at the Institute of Statistical Mathematics, Tokyo, between 11/2005 and 3/2007 and then in the financial industry between 4/2007 and 9/2008. After working at the ORFE department of Princeton University as a lecturer between 2/2009 and 8/2010, he was at the Graduate School of Informatics of Kyoto University between 9/2010 and 9/2016 as a tenured associate professor. He joined ENSAE in 9/2016 as a professor, where he is now working part-time. He was at Google between 10/2018 and 1/2022. His main employment is now with Apple, since 1/2022, as a research scientist working on fundamental aspects of machine learning.
More from the Same Authors
-
2021 : Linear-Time Gromov Wasserstein Distances using Low Rank Couplings and Costs »
Meyer Scetbon · Gabriel Peyré · Marco Cuturi -
2021 : Linear-Time Gromov Wasserstein Distances using Low Rank Couplings and Costs »
Meyer Scetbon · Gabriel Peyré · Marco Cuturi -
2023 Poster: Physics-Informed Bayesian Optimization of Variational Quantum Circuits »
Kim Nicoli · Christopher Anders · Lena Funcke · Tobias Hartung · Karl Jansen · Stefan Kühn · Klaus-Robert Müller · Paolo Stornati · Pan Kessel · Shinichi Nakajima -
2023 Poster: Unbalanced Low-rank Optimal Transport Solvers »
Meyer Scetbon · Michal Klein · Giovanni Palla · Marco Cuturi -
2023 Workshop: Optimal Transport and Machine Learning »
Anna Korba · Aram-Alexandre Pooladian · Charlotte Bunne · David Alvarez-Melis · Marco Cuturi · Ziv Goldfeld -
2022 Poster: Supervised Training of Conditional Monge Maps »
Charlotte Bunne · Andreas Krause · Marco Cuturi -
2022 Poster: Efficient and Modular Implicit Differentiation »
Mathieu Blondel · Quentin Berthet · Marco Cuturi · Roy Frostig · Stephan Hoyer · Felipe Llinares-Lopez · Fabian Pedregosa · Jean-Philippe Vert -
2022 Poster: So3krates: Equivariant attention for interactions on arbitrary length-scales in molecular systems »
Thorben Frank · Oliver Unke · Klaus-Robert Müller -
2022 Poster: Low-rank Optimal Transport: Approximation, Statistics and Debiasing »
Meyer Scetbon · Marco Cuturi -
2021 Workshop: Optimal Transport and Machine Learning »
Jason Altschuler · Charlotte Bunne · Laetitia Chapel · Marco Cuturi · Rémi Flamary · Gabriel Peyré · Alexandra Suvorikova -
2021 Poster: Efficient hierarchical Bayesian inference for spatio-temporal regression models in neuroimaging »
Ali Hashemi · Yijing Gao · Chang Cai · Sanjay Ghosh · Klaus-Robert Müller · Srikantan Nagarajan · Stefan Haufe -
2021 Poster: SE(3)-equivariant prediction of molecular wavefunctions and electronic densities »
Oliver Unke · Mihail Bogojeski · Michael Gastegger · Mario Geiger · Tess Smidt · Klaus-Robert Müller -
2020 : Panel »
Alan Aspuru-Guzik · Jennifer Listgarten · Klaus-Robert Müller · Nadine Schneider -
2020 : Invited Talk: Klaus Robert-Müller & Kristof Schütt: Machine Learning meets Quantum Chemistry »
Klaus-Robert Müller · Kristof Schütt -
2020 Poster: Projection Robust Wasserstein Distance and Riemannian Optimization »
Tianyi Lin · Chenyou Fan · Nhat Ho · Marco Cuturi · Michael Jordan -
2020 Poster: Fixed-Support Wasserstein Barycenters: Computational Hardness and Fast Algorithm »
Tianyi Lin · Nhat Ho · Xi Chen · Marco Cuturi · Michael Jordan -
2020 Spotlight: Projection Robust Wasserstein Distance and Riemannian Optimization »
Tianyi Lin · Chenyou Fan · Nhat Ho · Marco Cuturi · Michael Jordan -
2020 Poster: Learning with Differentiable Pertubed Optimizers »
Quentin Berthet · Mathieu Blondel · Olivier Teboul · Marco Cuturi · Jean-Philippe Vert · Francis Bach -
2020 Poster: Entropic Optimal Transport between Unbalanced Gaussian Measures has a Closed Form »
Hicham Janati · Boris Muzellec · Gabriel Peyré · Marco Cuturi -
2020 Poster: Linear Time Sinkhorn Divergences using Positive Features »
Meyer Scetbon · Marco Cuturi -
2020 Oral: Entropic Optimal Transport between Unbalanced Gaussian Measures has a Closed Form »
Hicham Janati · Boris Muzellec · Gabriel Peyré · Marco Cuturi -
2020 Session: Orals & Spotlights Track 21: Optimization »
Peter Richtarik · Marco Cuturi -
2019 Workshop: Optimal Transport for Machine Learning »
Marco Cuturi · Gabriel Peyré · Rémi Flamary · Alexandra Suvorikova -
2019 Poster: Subspace Detours: Building Transport Plans that are Optimal on Subspace Projections »
Boris Muzellec · Marco Cuturi -
2019 Poster: Differentiable Ranking and Sorting using Optimal Transport »
Marco Cuturi · Olivier Teboul · Jean-Philippe Vert -
2019 Spotlight: Differentiable Ranking and Sorting using Optimal Transport »
Marco Cuturi · Olivier Teboul · Jean-Philippe Vert -
2019 Demonstration: Learning Machines can Curl - Adaptive Deep Reinforcement Learning enables the robot Curly to win against human players in an icy world »
Dong-Ok Won · Sang-Hoon Lee · Klaus-Robert Müller · Seong-Whan Lee -
2019 Poster: Tree-Sliced Variants of Wasserstein Distances »
Tam Le · Makoto Yamada · Kenji Fukumizu · Marco Cuturi -
2019 Poster: Explanations can be manipulated and geometry is to blame »
Ann-Kathrin Dombrowski · Maximillian Alber · Christopher Anders · Marcel Ackermann · Klaus-Robert Müller · Pan Kessel -
2018 Workshop: Machine Learning for Molecules and Materials »
José Miguel Hernández-Lobato · Klaus-Robert Müller · Brooks Paige · Matt Kusner · Stefan Chmiela · Kristof Schütt -
2018 Poster: Large Scale computation of Means and Clusters for Persistence Diagrams using Optimal Transport »
Theo Lacombe · Marco Cuturi · Steve OUDOT -
2018 Poster: Generalizing Point Embeddings using the Wasserstein Space of Elliptical Distributions »
Boris Muzellec · Marco Cuturi -
2017 : Methods 1 »
Grégoire Montavon · Michael Tsang · Marco Ancona -
2017 : Opening Remarks »
Klaus-Robert Müller -
2017 Workshop: Optimal Transport and Machine Learning »
Olivier Bousquet · Marco Cuturi · Gabriel Peyré · Fei Sha · Justin Solomon -
2017 Workshop: Interpreting, Explaining and Visualizing Deep Learning - Now what ? »
Klaus-Robert Müller · Andrea Vedaldi · Lars K Hansen · Wojciech Samek · Grégoire Montavon -
2017 Workshop: Machine Learning for Molecules and Materials »
Kristof Schütt · Klaus-Robert Müller · Anatole von Lilienfeld · José Miguel Hernández-Lobato · Klaus-Robert Müller · Alan Aspuru-Guzik · Bharath Ramsundar · Matt Kusner · Brooks Paige · Stefan Chmiela · Alexandre Tkatchenko · Anatole von Lilienfeld · Koji Tsuda -
2017 : Opening remarks »
Klaus-Robert Müller -
2017 Poster: SchNet: A continuous-filter convolutional neural network for modeling quantum interactions »
Kristof Schütt · Pieter-Jan Kindermans · Huziel Enoc Sauceda Felix · Stefan Chmiela · Alexandre Tkatchenko · Klaus-Robert Müller -
2017 Poster: An Empirical Study on The Properties of Random Bases for Kernel Methods »
Maximilian Alber · Pieter-Jan Kindermans · Kristof Schütt · Klaus-Robert Müller · Fei Sha -
2017 Tutorial: A Primer on Optimal Transport »
Marco Cuturi · Justin Solomon -
2016 Workshop: Time Series Workshop »
Oren Anava · Marco Cuturi · Azadeh Khaleghi · Vitaly Kuznetsov · Sasha Rakhlin -
2016 Poster: Stochastic Optimization for Large-scale Optimal Transport »
Aude Genevay · Marco Cuturi · Gabriel Peyré · Francis Bach -
2015 : Explaining individual deep network predictions and measuring the quality of these explanations »
Grégoire Montavon -
2015 Poster: Principal Geodesic Analysis for Probability Measures under the Optimal Transport Metric »
Vivien Seguy · Marco Cuturi -
2014 Workshop: Optimal Transport and Machine Learning »
Marco Cuturi · Gabriel Peyré · Justin Solomon · Alexander Barvinok · Piotr Indyk · Robert McCann · Adam Oberman -
2014 Poster: Covariance shrinkage for autocorrelated data »
Daniel Bartz · Klaus-Robert Müller -
2013 Poster: Robust Spatial Filtering with Beta Divergence »
Wojciech Samek · Duncan Blythe · Klaus-Robert Müller · Motoaki Kawanabe -
2013 Poster: Generalizing Analytic Shrinkage for Arbitrary Covariance Structures »
Daniel Bartz · Klaus-Robert Müller -
2013 Poster: Sinkhorn Distances: Lightspeed Computation of Optimal Transport »
Marco Cuturi -
2013 Spotlight: Sinkhorn Distances: Lightspeed Computation of Optimal Transport »
Marco Cuturi -
2013 Spotlight: Robust Spatial Filtering with Beta Divergence »
Wojciech Samek · Duncan Blythe · Klaus-Robert Müller · Motoaki Kawanabe -
2013 Spotlight: Generalizing Analytic Shrinkage for Arbitrary Covariance Structures »
Daniel Bartz · Klaus-Robert Müller -
2012 Poster: Learning Invariant Representations of Molecules for Atomization Energy Prediction »
Grégoire Montavon · Katja Hansen · Siamac Fazli · Matthias Rupp · Franziska Biegler · Andreas Ziehe · Alexandre Tkatchenko · Anatole von Lilienfeld · Klaus-Robert Müller -
2011 Demonstration: Real-time social media analysis with TWIMPACT »
Mikio L Braun · Matthias L Jugel · Klaus-Robert Müller -
2010 Workshop: Charting Chemical Space: Challenges and Opportunities for AI and Machine Learning »
Pierre Baldi · Klaus-Robert Müller · Gisbert Schneider -
2010 Poster: Layer-wise analysis of deep networks with Gaussian kernels »
Grégoire Montavon · Mikio L Braun · Klaus-Robert Müller -
2009 Poster: Efficient and Accurate Lp-Norm Multiple Kernel Learning »
Marius Kloft · Ulf Brefeld · Soeren Sonnenburg · Pavel Laskov · Klaus-Robert Müller · Alexander Zien -
2009 Poster: Subject independent EEG-based BCI decoding »
Siamac Fazli · Cristian Grozea · Márton Danóczy · Benjamin Blankertz · Florin Popescu · Klaus-Robert Müller -
2009 Spotlight: Subject independent EEG-based BCI decoding »
Siamac Fazli · Cristian Grozea · Márton Danóczy · Benjamin Blankertz · Florin Popescu · Klaus-Robert Müller -
2009 Poster: White Functionals for Anomaly Detection in Dynamical Systems »
Marco Cuturi · Jean-Philippe Vert · Alexandre d'Aspremont -
2008 Poster: Playing Pinball with non-invasive BCI »
Michael W Tangermann (ne Schröder) · Matthias Krauledat · Konrad Grzeska · Max Sagebaum · Benjamin Blankertz · Klaus-Robert Müller -
2008 Poster: Estimating vector fields using sparse basis field expansions »
Stefan Haufe · Vadim Nikulin · Andreas Ziehe · Klaus-Robert Müller · Guido Nolte -
2007 Spotlight: Invariant Common Spatial Patterns: Alleviating Nonstationarities in Brain-Computer Interfacing »
Benjamin Blankertz · Motoaki Kawanabe · Ryota Tomioka · Friederike Hohlefeld · Vadim Nikulin · Klaus-Robert Müller -
2007 Poster: Invariant Common Spatial Patterns: Alleviating Nonstationarities in Brain-Computer Interfacing »
Benjamin Blankertz · Motoaki Kawanabe · Ryota Tomioka · Friederike Hohlefeld · Vadim Nikulin · Klaus-Robert Müller -
2007 Poster: Heterogeneous Component Analysis »
Shigeyuki Oba · Motoaki Kawanabe · Klaus-Robert Müller · Shin Ishii -
2007 Spotlight: Heterogeneous Component Analysis »
Shigeyuki Oba · Motoaki Kawanabe · Klaus-Robert Müller · Shin Ishii -
2006 Workshop: Current Trends in Brain-Computer Interfacing »
Klaus-Robert Müller · José del R. Millán · Matthias Krauledat · Roderick Murray-Smith · Benjamin Blankertz -
2006 Poster: Logistic Regression for Single Trial EEG Classification »
Ryota Tomioka · Kazuyuki Aihara · Klaus-Robert Müller -
2006 Poster: Towards Zero-Training for Brain-Computer Interface Experiments »
Matthias Krauledat · Michael Schröder · Benjamin Blankertz · Klaus-Robert Müller -
2006 Poster: Kernels on Structured Objects Through Nested Histograms »
Marco Cuturi · Kenji Fukumizu -
2006 Spotlight: Logistic Regression for Single Trial EEG Classification »
Ryota Tomioka · Kazuyuki Aihara · Klaus-Robert Müller -
2006 Poster: Inducing Metric Violations in Human Similarity Judgements »
Julian Laub · Jakob H Macke · Klaus-Robert Müller · Felix A Wichmann -
2006 Poster: Denoising and Dimension Reduction in Feature Space »
Mikio L Braun · Joachim M Buhmann · Klaus-Robert Müller