Timezone: »

Multimodal Generative Learning Utilizing Jensen-Shannon-Divergence
Thomas Sutter · Imant Daunhawer · Julia Vogt

Thu Dec 10 09:00 AM -- 11:00 AM (PST) @ Poster Session 5 #1356

Learning from different data types is a long-standing goal in machine learning research, as multiple information sources co-occur when describing natural phenomena. However, existing generative models that approximate a multimodal ELBO rely on difficult or inefficient training schemes to learn a joint distribution and the dependencies between modalities. In this work, we propose a novel, efficient objective function that utilizes the Jensen-Shannon divergence for multiple distributions. It simultaneously approximates the unimodal and joint multimodal posteriors directly via a dynamic prior. In addition, we theoretically prove that the new multimodal JS-divergence (mmJSD) objective optimizes an ELBO. In extensive experiments, we demonstrate the advantage of the proposed mmJSD model compared to previous work in unsupervised, generative learning tasks.

Author Information

Thomas Sutter (ETH Zurich)
Imant Daunhawer (ETH Zurich)
Julia Vogt (ETH Zurich)

More from the Same Authors