Timezone: »

 
Multimodal Relational VAE
Thomas Sutter · Julia Vogt
Event URL: https://openreview.net/forum?id=DjFzMOOuh_w »

In this work, we propose a new formulation for multimodal VAEs to model and learn the relationship between data types. Despite their recent progress, current multimodal generative methods are based on simplistic assumptions regarding the relation between data types, which leads to a trade-off between coherence and quality of generated samples - even for simple toy datasets. The proposed method learns the relationship between data types instead of relying on pre-defined and limiting assumptions. Based on the principles of variational inference, we change the posterior approximation to explicitly include information about the relation between data types. We show empirically that the simplified assumption of a single shared latent space leads to inferior performance for a dataset with additional pairwise shared information.

Author Information

Thomas Sutter (ETH Zurich)
Julia Vogt (University of Basel)

More from the Same Authors

  • 2022 : M(otion)-mode Based Prediction of Cardiac Function on Echocardiograms »
    Thomas Sutter · Sebastian Balzer · Ece Ozkan · Julia Vogt
  • 2021 Poster: Deep Conditional Gaussian Mixture Model for Constrained Clustering »
    Laura Manduchi · Kieran Chin-Cheong · Holger Michel · Sven Wellmann · Julia Vogt
  • 2020 Poster: Multimodal Generative Learning Utilizing Jensen-Shannon-Divergence »
    Thomas Sutter · Imant Daunhawer · Julia Vogt
  • 2019 : Poster session »
    Candace Ross · Yassine Mrabet · Sanjay Subramanian · Geoffrey Cideron · Jesse Mu · Suvrat Bhooshan · Eda Okur Kavil · Jean-Benoit Delbrouck · Yen-Ling Kuo · Nicolas Lair · Gabriel Ilharco · T.S. Jayram · Alba MarĂ­a Herrera Palacio · Chihiro Fujiyama · Olivier Tieleman · Anna Potapenko · Guan-Lin Chao · Thomas Sutter · Olga Kovaleva · Farley Lai · Xin Wang · Vasu Sharma · Catalina Cangea · Nikhil Krishnaswamy · Yuta Tsuboi · Alexander Kuhnle · Khanh Nguyen · Dian Yu · Homagni Saha · Jiannan Xiang · Vijay Venkataraman · Ankita Kalra · Ning Xie · Derek Doran · Travis Goodwin · Asim Kadav · Shabnam Daghaghi · Jason Baldridge · Jialin Wu · Jingxiang Lin · Unnat Jain