Timezone: »

 
Poster
Learning in Markov Random Fields using Tempered Transitions
Russ Salakhutdinov

Tue Dec 08 07:00 PM -- 11:59 PM (PST) @
in Poster Session »

Markov random fields (MRFs), or undirected graphical models, provide a powerful framework for modeling complex dependencies among random variables. Maximum likelihood learning in MRFs is hard due to the presence of the global normalizing constant. In this paper we consider a class of stochastic approximation algorithms of Robbins-Monro type that uses Markov chain Monte Carlo to do approximate maximum likelihood learning. We show that using MCMC operators based on tempered transitions enables the stochastic approximation algorithm to better explore highly multimodal distributions, which considerably improves parameter estimates in large densely-connected MRFs. Our results on MNIST and NORB datasets demonstrate that we can successfully learn good generative models of high-dimensional, richly structured data and perform well on digit and object recognition tasks.

[ PDF] 

Author Information

Russ Salakhutdinov (Carnegie Mellon University)

More from the Same Authors