Timezone: »

Invariant Feature Subspace Recovery for Multi-Class Classification
Gargi Balasubramaniam · Haoxiang Wang · Han Zhao

Domain generalization aims to learn a model over multiple training environments to generalize to unseen environments. Recently, Wang et al [2022] proposed Invariant-feature Subspace Recovery (ISR), a domain generalization algorithm which uses the means of class-conditional data distributions to provably identify the invariant-feature subspace. However, the original ISR algorithm is conditioned on single class only, without utilizing information from the rest classes. In this work, we consider the setting of multi-class classification, and propose an extension of the ISR algorithm, called ISR-Multiclass. This proposed algorithm can provably recover the invariant-feature subspace with $\mathcal{O}(d_{spu}/k) + 1$ environments, where $d_{spu}$ is the number of spurious features and $k$ is the number of classes. Empirically, we first examine ISR-Multiclass in a synthetic dataset, and demonstrate its superiority over the original ISR in the multi-class setting. Furthermore, we conduct experiments in Multiclass Coloured MNIST, a semi-synthetic dataset with strong spurious correlations, and show that ISR-Multiclass can significantly improve the robustness of neural nets trained by various methods (e.g., ERM and IRM) against spurious correlations.

#### Author Information

##### Haoxiang Wang (University of Illinois, Urbana-Champaign)

1st year PhD student working on machine learning from UIUC. Has one Spotlight paper at NeurIPS 2019: https://papers.nips.cc/paper/9563-learning-positive-functions-with-pseudo-mirror-descent CV: https://www.dropbox.com/s/1w7iaw1h1x8yhap/Haoxiang_Wang_UIUC-CV.pdf?dl=0