Timezone: »

 
Poster
Alleviating "Posterior Collapse'' in Deep Topic Models via Policy Gradient
Yewen Li · Chaojie Wang · Zhibin Duan · Dongsheng Wang · Bo Chen · Bo An · Mingyuan Zhou

Wed Nov 30 02:00 PM -- 04:00 PM (PST) @ Hall J #627
in Poster Session 4 »

Deep topic models have been proven as a promising way to extract hierarchical latent representations from documents represented as high-dimensional bag-of-words vectors.However, the representation capability of existing deep topic models is still limited by the phenomenon of "posterior collapse", which has been widely criticized in deep generative models, resulting in the higher-level latent representations exhibiting similar or meaningless patterns.To this end, in this paper, we first develop a novel deep-coupling generative process for existing deep topic models, which incorporates skip connections into the generation of documents, enforcing strong links between the document and its multi-layer latent representations.After that, utilizing data augmentation techniques, we reformulate the deep-coupling generative process as a Markov decision process and develop a corresponding Policy Gradient (PG) based training algorithm, which can further alleviate the information reduction at higher layers.Extensive experiments demonstrate that our developed methods can effectively alleviate "posterior collapse" in deep topic models, contributing to providing higher-quality latent document representations.

Keywords: Reinforcement Learning · posterior collapse · Deep Topic Models
[ Paper]  [ Poster]  [ OpenReview] 

Author Information

Yewen Li (nanyang technological university)
Chaojie Wang (Nanyang Technological University)
Zhibin Duan (Xidian University)
Dongsheng Wang (Xidian University)
Bo Chen (Xidian University)
Bo An (Nanyang Technological University)
Mingyuan Zhou (University of Texas at Austin)

More from the Same Authors