Timezone: »

 
Poster
Learning Causal Semantic Representation for Out-of-Distribution Prediction
Chang Liu · Xinwei Sun · Jindong Wang · Haoyue Tang · Tao Li · Tao Qin · Wei Chen · Tie-Yan Liu

Wed Dec 08 12:30 AM -- 02:00 AM (PST) @ Virtual
in Poster Session 3 »

Conventional supervised learning methods, especially deep ones, are found to be sensitive to out-of-distribution (OOD) examples, largely because the learned representation mixes the semantic factor with the variation factor due to their domain-specific correlation, while only the semantic factor causes the output. To address the problem, we propose a Causal Semantic Generative model (CSG) based on a causal reasoning so that the two factors are modeled separately, and develop methods for OOD prediction from a single training domain, which is common and challenging. The methods are based on the causal invariance principle, with a novel design in variational Bayes for both efficient learning and easy prediction. Theoretically, we prove that under certain conditions, CSG can identify the semantic factor by fitting training data, and this semantic-identification guarantees the boundedness of OOD generalization error and the success of adaptation. Empirical study shows improved OOD performance over prevailing baselines.

Keywords: Generative Model · Domain Adaptation · Representation Learning

Author Information

Chang Liu (Microsoft Research Asia)
Xinwei Sun (Peking University)
Jindong Wang (Microsoft Research Asia)
Haoyue Tang (Tsinghua University, Tsinghua University)
Tao Li (Peking University)
Tao Qin (Microsoft Research)
Wei Chen (Chinese Academy of Sciences)
Tie-Yan Liu (Microsoft Research Asia)

More from the Same Authors