Timezone: »

 
Poster
Delving into Out-of-Distribution Detection with Vision-Language Representations
Yifei Ming · Ziyang Cai · Jiuxiang Gu · Yiyou Sun · Wei Li · Yixuan Li

Tue Nov 29 09:00 AM -- 11:00 AM (PST) @ Hall J #137
in Poster Session 1 »

Recognizing out-of-distribution (OOD) samples is critical for machine learning systems deployed in the open world. The vast majority of OOD detection methods are driven by a single modality (e.g., either vision or language), leaving the rich information in multi-modal representations untapped. Inspired by the recent success of vision-language pre-training, this paper enriches the landscape of OOD detection from a single-modal to a multi-modal regime. Particularly, we propose Maximum Concept Matching (MCM), a simple yet effective zero-shot OOD detection method based on aligning visual features with textual concepts. We contribute in-depth analysis and theoretical insights to understand the effectiveness of MCM. Extensive experiments demonstrate that MCM achieves superior performance on a wide variety of real-world tasks. MCM with vision-language features outperforms a common baseline with pure visual features on a hard OOD task with semantically similar classes by 13.1% (AUROC) Code is available at https://github.com/deeplearning-wisc/MCM.

[ Paper]  [ Poster]  [ OpenReview] 

Author Information

Yifei Ming (University of Wisconsin-Madison)

I'm a Ph.D. student at the University of Wisconsin-Madison. I’m broadly interested in trustworthy machine learning and representation learning. Research topics that I am currently focusing on include: out-of-distribution detection, domain generalization, supervised and self-supervised (multi-modal) representation learning. My prior research involves designing efficient algorithms and promoting fundamental understandings to enable reliable open-world learning. (e.g., impact of spurious correlation, sample efficiency, and multi-modality).

Ziyang Cai (University of Wisconsin - Madison)
Jiuxiang Gu (Adobe Research)
Yiyou Sun (University of Wisconsin, Madison)
Wei Li (GOOGLE INC)
Yixuan Li (University of Wisconsin-Madison)

More from the Same Authors