Timezone: »

 
Poster
DualCoOp: Fast Adaptation to Multi-Label Recognition with Limited Annotations
Ximeng Sun · Ping Hu · Kate Saenko

Thu Dec 01 09:00 AM -- 11:00 AM (PST) @ Hall J #238
in Poster Session 5 »

Solving multi-label recognition (MLR) for images in the low-label regime is a challenging task with many real-world applications. Recent work learns an alignment between textual and visual spaces to compensate for insufficient image labels, but loses accuracy because of the limited amount of available MLR annotations. In this work, we utilize the strong alignment of textual and visual features pretrained with millions of auxiliary image-text pairs and propose \textit{Dual Context Optimization} (DualCoOp) as a unified framework for partial-label MLR and zero-shot MLR. \ours encodes positive and negative contexts with class names as part of the linguistic input (i.e. prompts). Since \ours only introduces a very light learnable overhead upon the pretrained vision-language framework, it can quickly adapt to multi-label recognition tasks that have limited annotations and even unseen classes. Experiments on standard multi-label recognition benchmarks across two challenging low-label settings demonstrate the advantages of our approach over state-of-the-art methods. Our code will be publicly available.Project page: https://cs-people.bu.edu/sunxm/DualCoOp/project.html

Keywords: Multi-Label Recognition · Prompt Learning
[ Paper]  [ Poster]  [ OpenReview] 

Author Information

Ximeng Sun (Boston University)
Ping Hu (Boston University)
Kate Saenko (Boston University & MIT-IBM Watson AI Lab, IBM Research)

More from the Same Authors