Timezone: »

 
Poster
Learning to Discover and Detect Objects
Vladimir Fomenko · Ismail Elezi · Deva Ramanan · Laura Leal-Taixé · Aljosa Osep

Wed Nov 30 09:00 AM -- 11:00 AM (PST) @ Hall J #230
in Poster Session 3 »

We tackle the problem of novel class discovery and localization (NCDL). In this setting, we assume a source dataset with supervision for only some object classes. Instances of other classes need to be discovered, classified, and localized automatically based on visual similarity without any human supervision. To tackle NCDL, we propose a two-stage object detection network Region-based NCDL (RNCDL) that uses a region proposal network to localize regions of interest (RoIs). We then train our network to learn to classify each RoI, either as one of the known classes, seen in the source dataset, or one of the novel classes, with a long-tail distribution constraint on the class assignments, reflecting the natural frequency of classes in the real world. By training our detection network with this objective in an end-to-end manner, it learns to classify all region proposals for a large variety of classes, including those not part of the labeled object class vocabulary. Our experiments conducted using COCO and LVIS datasets reveal that our method is significantly more effective than multi-stage pipelines that rely on traditional clustering algorithms. Furthermore, we demonstrate the generality of our approach by applying our method to a large-scale Visual Genome dataset, where our network successfully learns to detect various semantic classes without direct supervision.

Keywords: Representation Learning · transfer learning · Recognition and classification · Vision applications and systems · Detection and localization in 2D and/or 3D
[ Paper]  [ Poster]  [ OpenReview] 

Author Information

Vladimir Fomenko (Technical University of Munich)
Vladimir Fomenko

I currently conduct applied research and machine learning engineering for large language models as an applied scientist in Azure AI. Prior to this, I completed my master's degree in Artificial Intelligence from Technical University of Munich. My research interests are in natural language processing, computer vision, and representation learning, using advanced deep learning methods.

Ismail Elezi (Technical University of Munich)
Deva Ramanan (Carnegie Mellon University)
Laura Leal-Taixé (TUM)
Aljosa Osep (TU Munich)

More from the Same Authors