Timezone: »
Most existing works in few-shot learning rely on meta-learning the network on a large base dataset which is typically from the same domain as the target dataset. We tackle the problem of cross-domain few-shot learning where there is a large shift between the base and target domain. The problem of cross-domain few-shot recognition with unlabeled target data is largely unaddressed in the literature. STARTUP was the first method that tackles this problem using self-training. However, it uses a fixed teacher pretrained on a labeled base dataset to create soft labels for the unlabeled target samples. As the base dataset and unlabeled dataset are from different domains, projecting the target images in the class-domain of the base dataset with a fixed pretrained model might be sub-optimal. We propose a simple dynamic distillation-based approach to facilitate unlabeled images from the novel/base dataset. We impose consistency regularization by calculating predictions from the weakly-augmented versions of the unlabeled images from a teacher network and matching it with the strongly augmented versions of the same images from a student network. The parameters of the teacher network are updated as exponential moving average of the parameters of the student network. We show that the proposed network learns representation that can be easily adapted to the target domain even though it has not been trained with target-specific classes during the pretraining phase. Our model outperforms the current state-of-the art method by 4.4% for 1-shot and 3.6% for 5-shot classification in the BSCD-FSL benchmark, and also shows competitive performance on traditional in-domain few-shot learning task.
Author Information
Ashraful Islam (Rensselaer Polytechnic Institute)
I am a final year PhD student at the ECSE department of Rensselaer Polytechnic Institute (RPI), Troy, NY, supervised by Prof. Richard Radke. Broadly, I am interested in computer vision and deep learning. I develop neural network models for unsupervised, semi-supervised, weakly-supervised, and few-shot learning. I have also worked on action detection and object tracking. Previously, I spent three wonderful summers at Microsoft Research, IBM Research and Kitware. I completed my undergrad in Electrical Engineering from BUET, Bangladesh.
Chun-Fu (Richard) Chen (MIT-IBM Watson AI Lab)
Rameswar Panda (MIT-IBM Watson AI Lab)
Leonid Karlinsky (Weizmann Institute of Science)
Rogerio Feris (MIT-IBM Watson AI Lab, IBM Research)
Richard J. Radke (Rensselaer Polytechnic Institute)
More from the Same Authors
-
2021 : Select, Label, and Mix: Learning Discriminative Invariant Feature Representations for Partial Domain Adaptation »
Aadarsh Sahoo · Rameswar Panda · Rogerio Feris · Kate Saenko · Abir Das -
2023 Poster: LaFTer: Label-Free Tuning of Zero-shot Classifier using Language and Unlabeled Image Collections »
Muhammad Jehanzeb Mirza · Leonid Karlinsky · Wei Lin · Horst Possegger · Mateusz Kozinski · Rogerio Feris · Horst Bischof -
2023 Poster: Dense and Aligned Captions (DAC) Promote Compositional Reasoning in VL Models »
Sivan Doveh · Assaf Arbelle · Sivan Harary · Roei Herzig · Donghyun Kim · Paola Cascante-Bonilla · Amit Alfassy · Rameswar Panda · Raja Giryes · Rogerio Feris · Shimon Ullman · Leonid Karlinsky -
2023 Poster: Learning Human Action Recognition Representations Without Real Humans »
Howard Zhong · Samarth Mishra · Donghyun Kim · SouYoung Jin · Rameswar Panda · Hilde Kuehne · Leonid Karlinsky · Venkatesh Saligrama · Aude Oliva · Rogerio Feris -
2022 Poster: Procedural Image Programs for Representation Learning »
Manel Baradad · Richard Chen · Jonas Wulff · Tongzhou Wang · Rogerio Feris · Antonio Torralba · Phillip Isola -
2022 Poster: Bringing Image Scene Structure to Video via Frame-Clip Consistency of Object Tokens »
Elad Ben Avraham · Roei Herzig · Karttikeya Mangalam · Amir Bar · Anna Rohrbach · Leonid Karlinsky · Trevor Darrell · Amir Globerson -
2022 Poster: ConfLab: A Data Collection Concept, Dataset, and Benchmark for Machine Analysis of Free-Standing Social Interactions in the Wild »
Chirag Raman · Jose Vargas Quiros · Stephanie Tan · Ashraful Islam · Ekin Gedik · Hayley Hung -
2022 Poster: How Transferable are Video Representations Based on Synthetic Data? »
Yo-whan Kim · Samarth Mishra · SouYoung Jin · Rameswar Panda · Hilde Kuehne · Leonid Karlinsky · Venkatesh Saligrama · Kate Saenko · Aude Oliva · Rogerio Feris -
2022 Poster: FETA: Towards Specializing Foundational Models for Expert Task Applications »
Amit Alfassy · Assaf Arbelle · Oshri Halimi · Sivan Harary · Roei Herzig · Eli Schwartz · Rameswar Panda · Michele Dolfi · Christoph Auer · Peter Staar · Kate Saenko · Rogerio Feris · Leonid Karlinsky -
2021 Poster: IA-RED$^2$: Interpretability-Aware Redundancy Reduction for Vision Transformers »
Bowen Pan · Rameswar Panda · Yifan Jiang · Zhangyang Wang · Rogerio Feris · Aude Oliva -
2021 Poster: Contrast and Mix: Temporal Contrastive Video Domain Adaptation with Background Mixing »
Aadarsh Sahoo · Rutav Shah · Rameswar Panda · Kate Saenko · Abir Das -
2020 Poster: AdaShare: Learning What To Share For Efficient Deep Multi-Task Learning »
Ximeng Sun · Rameswar Panda · Rogerio Feris · Kate Saenko -
2019 : Adaptive Multi-Task Neural Networks for Efficient Inference »
Rogerio Feris -
2019 Poster: More Is Less: Learning Efficient Video Representations by Big-Little Network and Depthwise Temporal Aggregation »
Quanfu Fan · Chun-Fu (Richard) Chen · Hilde Kuehne · Marco Pistoia · David Cox -
2018 Poster: Delta-encoder: an effective sample synthesis method for few-shot object recognition »
Eli Schwartz · Leonid Karlinsky · Joseph Shtok · Sivan Harary · Mattias Marder · Abhishek Kumar · Rogerio Feris · Raja Giryes · Alex Bronstein -
2018 Spotlight: Delta-encoder: an effective sample synthesis method for few-shot object recognition »
Eli Schwartz · Leonid Karlinsky · Joseph Shtok · Sivan Harary · Mattias Marder · Abhishek Kumar · Rogerio Feris · Raja Giryes · Alex Bronstein -
2018 Poster: Dialog-based Interactive Image Retrieval »
Xiaoxiao Guo · Hui Wu · Yu Cheng · Steven Rennie · Gerald Tesauro · Rogerio Feris -
2018 Poster: Co-regularized Alignment for Unsupervised Domain Adaptation »
Abhishek Kumar · Prasanna Sattigeri · Kahini Wadhawan · Leonid Karlinsky · Rogerio Feris · Bill Freeman · Gregory Wornell -
2010 Poster: Using body-anchored priors for identifying actions in single images »
Leonid Karlinsky · Michael Dinerstein · Shimon Ullman