Timezone: »
Multi-View Representation Learning (MVRL) aims to discover a shared representation of observations from different views with the complex underlying correlation. In this paper, we propose a variational approach which casts MVRL as maximizing the amount of total correlation reduced by the representation, aiming to learn a shared latent representation that is informative yet succinct to capture the correlation among multiple views. To this end, we introduce a tractable surrogate objective function under the proposed framework, which allows our method to fuse and calibrate the observations in the representation space. From the information-theoretic perspective, we show that our framework subsumes existing multi-view generative models. Lastly, we show that our approach straightforwardly extends to the Partial MVRL (PMVRL) setting, where the observations are missing without any regular pattern. We demonstrate the effectiveness of our approach in the multi-view translation and classification tasks, outperforming strong baseline methods.
Author Information
HyeongJoo Hwang (KAIST)
Geon-Hyeong Kim (KAIST)
Seunghoon Hong (University of Michigan)
Kee-Eung Kim (KAIST)
More from the Same Authors
-
2023 Poster: Regularized Behavior Cloning for Blocking the Leakage of Past Action Information »
Seokin Seo · HyeongJoo Hwang · Hongseok Yang · Kee-Eung Kim -
2023 Poster: Addressing Out-Of-Distribution Joint Actions in Offline Multi-Agent RL via Alternating Stationary Distribution Correction Estimation »
Daiki E Matsunaga · Jongmin Lee · Jaeseok Yoon · Stefanos Leonardos · Pieter Abbeel · Kee-Eung Kim -
2022 Poster: LobsDICE: Offline Learning from Observation via Stationary Distribution Correction Estimation »
Geon-Hyeong Kim · Jongmin Lee · Youngsoo Jang · Hongseok Yang · Kee-Eung Kim -
2022 Poster: Local Metric Learning for Off-Policy Evaluation in Contextual Bandits with Continuous Actions »
Haanvid Lee · Jongmin Lee · Yunseon Choi · Wonseok Jeon · Byung-Jun Lee · Yung-Kyun Noh · Kee-Eung Kim -
2021 Poster: Transformers Generalize DeepSets and Can be Extended to Graphs & Hypergraphs »
Jinwoo Kim · Saeyoon Oh · Seunghoon Hong -
2020 Poster: Variational Interaction Information Maximization for Cross-domain Disentanglement »
HyeongJoo Hwang · Geon-Hyeong Kim · Seunghoon Hong · Kee-Eung Kim -
2018 Poster: Monte-Carlo Tree Search for Constrained POMDPs »
Jongmin Lee · Geon-Hyeong Kim · Pascal Poupart · Kee-Eung Kim -
2018 Poster: Learning Hierarchical Semantic Image Manipulation through Structured Representations »
Seunghoon Hong · Xinchen Yan · Thomas Huang · Honglak Lee