Timezone: »

 
Poster
Invariance Learning based on Label Hierarchy
Shoji Toyota · Kenji Fukumizu

Thu Dec 01 09:00 AM -- 11:00 AM (PST) @ Hall J #235
in Poster Session 5 »

Deep Neural Networks inherit spurious correlations embedded in training data and hence may fail to predict desired labels on unseen domains (or environments), which have different distributions from the domain to provide training data. Invariance Learning (IL) has been developed recently to overcome this shortcoming; using training data in many domains, IL estimates such a predictor that is invariant to a change of domain. However, the requirement of training data in multiple domains is a strong restriction of using IL, since it demands expensive annotation. We propose a novel IL framework to overcome this problem. Assuming the availability of data from multiple domains for a higher level of classification task, for which the labeling cost is lower, we estimate an invariant predictor for the target classification task with training data gathered in a single domain. Additionally, we propose two cross-validation methods for selecting hyperparameters of invariance regularization, which has not been addressed properly in existing IL methods. The effectiveness of the proposed framework, including the cross-validation, is demonstrated empirically. Theoretical analysis reveals that our framework can estimate the desirable invariant predictor with a hyperparameter fixed correctly, and that such a preferable hyperparameter is chosen by the proposed CV methods under some conditions.

Keywords: Deep Learning · Invariance · out-of-distribution generalization
[ Paper]  [ Poster]  [ OpenReview] 

Author Information

Shoji Toyota (The Graduate University for Advanced Studies, SOKENDAI)
Kenji Fukumizu (Institute of Statistical Mathematics / Preferred Networks / RIKEN AIP)

More from the Same Authors