Skip to yearly menu bar Skip to main content


Wasserstein Training of Restricted Boltzmann Machines

Grégoire Montavon · Klaus-Robert Müller · Marco Cuturi

Area 5+6+7+8 #137

Keywords: [ (Other) Probabilistic Models and Methods ] [ (Other) Unsupervised Learning Methods ] [ Deep Learning or Neural Networks ]


Boltzmann machines are able to learn highly complex, multimodal, structured and multiscale real-world data distributions. Parameters of the model are usually learned by minimizing the Kullback-Leibler (KL) divergence from training samples to the learned model. We propose in this work a novel approach for Boltzmann machine training which assumes that a meaningful metric between observations is given. This metric can be represented by the Wasserstein distance between distributions, for which we derive a gradient with respect to the model parameters. Minimization of this new objective leads to generative models with different statistical properties. We demonstrate their practical potential on data completion and denoising, for which the metric between observations plays a crucial role.

Live content is unavailable. Log in and register to view live content