Timezone: »

 
Poster
Trustworthy Multimodal Regression with Mixture of Normal-inverse Gamma Distributions
Huan Ma · Zongbo Han · Changqing Zhang · Huazhu Fu · Joey Tianyi Zhou · Qinghua Hu

Thu Dec 09 12:30 AM -- 02:00 AM (PST) @
in Poster Session 5 »

Multimodal regression is a fundamental task, which integrates the information from different sources to improve the performance of follow-up applications. However, existing methods mainly focus on improving the performance and often ignore the confidence of prediction for diverse situations. In this study, we are devoted to trustworthy multimodal regression which is critical in cost-sensitive domains. To this end, we introduce a novel Mixture of Normal-Inverse Gamma distributions (MoNIG) algorithm, which efficiently estimates uncertainty in principle for adaptive integration of different modalities and produces a trustworthy regression result. Our model can be dynamically aware of uncertainty for each modality, and also robust for corrupted modalities. Furthermore, the proposed MoNIG ensures explicitly representation of (modality-specific/global) epistemic and aleatoric uncertainties, respectively. Experimental results on both synthetic and different real-world data demonstrate the effectiveness and trustworthiness of our method on various multimodal regression tasks (e.g., temperature prediction for superconductivity, relative location prediction for CT slices, and multimodal sentiment analysis).

Keywords:

Author Information

Huan Ma (Tianjin University)
Zongbo Han (Tianjin University)
Changqing Zhang (Tianjin University)
Huazhu Fu (Inception Institute of Artificial Intelligence)
Joey Tianyi Zhou (IHPC, A*STAR)
Qinghua Hu (Tianjin University)

More from the Same Authors