Skip to yearly menu bar Skip to main content


Variational Bayes on Monte Carlo Steroids

Aditya Grover · Stefano Ermon

Area 5+6+7+8 #68

Keywords: [ Graphical Models ] [ Variational Inference ]


Variational approaches are often used to approximate intractable posteriors or normalization constants in hierarchical latent variable models. While often effective in practice, it is known that the approximation error can be arbitrarily large. We propose a new class of bounds on the marginal log-likelihood of directed latent variable models. Our approach relies on random projections to simplify the posterior. In contrast to standard variational methods, our bounds are guaranteed to be tight with high probability. We provide a new approach for learning latent variable models based on optimizing our new bounds on the log-likelihood. We demonstrate empirical improvements on benchmark datasets in vision and language for sigmoid belief networks, where a neural network is used to approximate the posterior.

Live content is unavailable. Log in and register to view live content