Timezone: »

 
Poster
Langevin Autoencoders for Learning Deep Latent Variable Models
Shohei Taniguchi · Yusuke Iwasawa · Wataru Kumagai · Yutaka Matsuo

Thu Dec 01 09:00 AM -- 11:00 AM (PST) @ Hall J #221
in Poster Session 5 »

Markov chain Monte Carlo (MCMC), such as Langevin dynamics, is valid for approximating intractable distributions. However, its usage is limited in the context of deep latent variable models owing to costly datapoint-wise sampling iterations and slow convergence. This paper proposes the amortized Langevin dynamics (ALD), wherein datapoint-wise MCMC iterations are entirely replaced with updates of an encoder that maps observations into latent variables. This amortization enables efficient posterior sampling without datapoint-wise iterations. Despite its efficiency, we prove that ALD is valid as an MCMC algorithm, whose Markov chain has the target posterior as a stationary distribution under mild assumptions. Based on the ALD, we also present a new deep latent variable model named the Langevin autoencoder (LAE). Interestingly, the LAE can be implemented by slightly modifying the traditional autoencoder. Using multiple synthetic datasets, we first validate that ALD can properly obtain samples from target posteriors. We also evaluate the LAE on the image generation task, and show that our LAE can outperform existing methods based on variational inference, such as the variational autoencoder, and other MCMC-based methods in terms of the test likelihood.

Keywords: amortized inference · deep latent variable models · Langevin dynamics
[ Paper]  [ Slides]  [ Poster]  [ OpenReview] 

Author Information

Shohei Taniguchi (The University of Tokyo)
Yusuke Iwasawa (The University of Tokyo)
Wataru Kumagai (RIKEN)
Yutaka Matsuo (University of Tokyo)

More from the Same Authors