Timezone: »
Object detection has achieved substantial progress in the last decade. However, detecting novel classes with only few samples remains challenging, since deep learning under low data regime usually leads to a degraded feature space. Existing works employ a holistic fine-tuning paradigm to tackle this problem, where the model is first pre-trained on all base classes with abundant samples, and then it is used to carve the novel class feature space. Nonetheless, this paradigm is still imperfect. Durning fine-tuning, a novel class may implicitly leverage the knowledge of multiple base classes to construct its feature space, which induces a scattered feature space, hence violating the inter-class separability. To overcome these obstacles, we propose a two-step fine-tuning framework, Few-shot object detection via Association and DIscrimination (FADI), which builds up a discriminative feature space for each novel class with two integral steps. 1) In the association step, in contrast to implicitly leveraging multiple base classes, we construct a compact novel class feature space via explicitly imitating a specific base class feature space. Specifically, we associate each novel class with a base class according to their semantic similarity. After that, the feature space of a novel class can readily imitate the well-trained feature space of the associated base class. 2) In the discrimination step, to ensure the separability between the novel classes and associated base classes, we disentangle the classification branches for base and novel classes. To further enlarge the inter-class separability between all classes, a set-specialized margin loss is imposed. Extensive experiments on standard Pascal VOC and MS-COCO datasets demonstrate that FADI achieves new state-of-the-art performance, significantly improving the baseline in any shot/split by +18.7. Notably, the advantage of FADI is most announced on extremely few-shot scenarios (e.g. 1- and 3- shot).
Author Information
Yuhang Cao (The Chinese University of Hong Kong)
Jiaqi Wang (The Chinese University of Hong Kong)
Ying Jin (Tsinghua University)
Tong Wu (The Chinese University of Hong Kong)
Kai Chen (The Chinese University of Hong Kong)
Ziwei Liu (Nanyang Technological University)
Dahua Lin (The Chinese University of Hong Kong)
More from the Same Authors
-
2021 : Semi-Supervised Domain Generalization with Stochastic StyleMatch »
Kaiyang Zhou · Chen Change Loy · Ziwei Liu -
2022 Poster: Audio-Driven Co-Speech Gesture Video Generation »
Xian Liu · Qianyi Wu · Hang Zhou · Yuanqi Du · Wayne Wu · Dahua Lin · Ziwei Liu -
2022 Poster: Benchmarking and Analyzing 3D Human Pose and Shape Estimation Beyond Algorithms »
Hui En Pang · Zhongang Cai · Lei Yang · Tianwei Zhang · Ziwei Liu -
2022 : Sparse Mixture-of-Experts are Domain Generalizable Learners »
Bo Li · Yifei Shen · Jingkang Yang · Yezhen Wang · Jiawei Ren · Tong Che · Jun Zhang · Ziwei Liu -
2023 Poster: What Makes Good Examples for Visual In-Context Learning? »
Yuanhan Zhang · Kaiyang Zhou · Ziwei Liu -
2023 Poster: Segment Any Point Cloud Sequences by Distilling Vision Foundation Models »
Youquan Liu · Lingdong Kong · Jun CEN · Runnan Chen · Wenwei Zhang · Liang Pan · Kai Chen · Ziwei Liu -
2023 Poster: InsActor: Instruction-driven Physics-based Characters »
Jiawei Ren · Mingyuan Zhang · Cunjun Yu · Xiao Ma · Liang Pan · Ziwei Liu -
2023 Poster: Towards Robust and Expressive Whole-body Human Pose and Shape Estimation »
Hui En Pang · Zhongang Cai · Lei Yang · Tianwei Zhang · Qingyi Tao · Zhonghua Wu · Ziwei Liu -
2023 Poster: FineMoGen: Fine-Grained Spatio-Temporal Motion Generation and Editing »
Mingyuan Zhang · Huirong Li · Zhongang Cai · Jiawei Ren · Lei Yang · Ziwei Liu -
2023 Poster: 4D Panoptic Scene Graph Generation »
Jingkang Yang · Jun CEN · WENXUAN PENG · Shuai Liu · Fangzhou Hong · Xiangtai Li · Kaiyang Zhou · Qifeng Chen · Ziwei Liu -
2023 Poster: PrimDiffusion: Volumetric Primitives Diffusion for 3D Human Generation »
Zhaoxi Chen · Fangzhou Hong · Haiyi Mei · Guangcong Wang · Lei Yang · Ziwei Liu -
2023 Poster: Language Models are Visual Reasoning Coordinators »
Liangyu Chen · Bo Li · Sheng Shen · Jingkang Yang · Chunyuan Li · Kurt Keutzer · Trevor Darrell · Ziwei Liu -
2023 Poster: SMPLer-X: Scaling Up Expressive Human Pose and Shape Estimation »
Zhongang Cai · Wanqi Yin · Ailing Zeng · CHEN WEI · Qingping SUN · Yanjun Wang · Hui En Pang · Haiyi Mei · Mingyuan Zhang · Lei Zhang · Chen Change Loy · Lei Yang · Ziwei Liu -
2023 Poster: RenderMe-360: A Large Digital Asset Library and Benchmarks Towards High-fidelity Head Avatars »
Dongwei Pan · Long Zhuo · Jingtan Piao · Huiwen Luo · Wei Cheng · Yuxin WANG · Siming Fan · Shengqi Liu · Lei Yang · Bo Dai · Ziwei Liu · Chen Change Loy · Chen Qian · Wayne Wu · Dahua Lin · Kwan-Yee Lin -
2022 : Factor Investing with a Deep Multi-Factor Model »
Zikai Wei · Bo Dai · Dahua Lin -
2022 Spotlight: Lightning Talks 4B-4 »
Ziyue Jiang · Zeeshan Khan · Yuxiang Yang · Chenze Shao · Yichong Leng · Zehao Yu · Wenguan Wang · Xian Liu · Zehua Chen · Yang Feng · Qianyi Wu · James Liang · C.V. Jawahar · Junjie Yang · Zhe Su · Songyou Peng · Yufei Xu · Junliang Guo · Michael Niemeyer · Hang Zhou · Zhou Zhao · Makarand Tapaswi · Dongfang Liu · Qian Yang · Torsten Sattler · Yuanqi Du · Haohe Liu · Jing Zhang · Andreas Geiger · Yi Ren · Long Lan · Jiawei Chen · Wayne Wu · Dahua Lin · Dacheng Tao · Xu Tan · Jinglin Liu · Ziwei Liu · 振辉 叶 · Danilo Mandic · Lei He · Xiangyang Li · Tao Qin · sheng zhao · Tie-Yan Liu -
2022 Spotlight: Audio-Driven Co-Speech Gesture Video Generation »
Xian Liu · Qianyi Wu · Hang Zhou · Yuanqi Du · Wayne Wu · Dahua Lin · Ziwei Liu -
2022 Spotlight: Benchmarking and Analyzing 3D Human Pose and Shape Estimation Beyond Algorithms »
Hui En Pang · Zhongang Cai · Lei Yang · Tianwei Zhang · Ziwei Liu -
2022 Poster: Semi-Supervised Semantic Segmentation via Gentle Teaching Assistant »
Ying Jin · Jiaqi Wang · Dahua Lin -
2022 Poster: AnimeRun: 2D Animation Visual Correspondence from Open Source 3D Movies »
Li Siyao · Yuhang Li · Bo Li · Chao Dong · Ziwei Liu · Chen Change Loy -
2022 Poster: OpenOOD: Benchmarking Generalized Out-of-Distribution Detection »
Jingkang Yang · Pengyun Wang · Dejian Zou · Zitang Zhou · Kunyuan Ding · WENXUAN PENG · Haoqi Wang · Guangyao Chen · Bo Li · Yiyou Sun · Xuefeng Du · Kaiyang Zhou · Wayne Zhang · Dan Hendrycks · Yixuan Li · Ziwei Liu -
2021 Poster: K-Net: Towards Unified Image Segmentation »
Wenwei Zhang · Jiangmiao Pang · Kai Chen · Chen Change Loy -
2021 Poster: Generative Occupancy Fields for 3D Surface-Aware Image Synthesis »
Xudong XU · Xingang Pan · Dahua Lin · Bo Dai -
2021 Poster: Balanced Chamfer Distance as a Comprehensive Metric for Point Cloud Completion »
Tong Wu · Liang Pan · Junzhe Zhang · Tai WANG · Ziwei Liu · Dahua Lin -
2021 Poster: Garment4D: Garment Reconstruction from Point Cloud Sequences »
Fangzhou Hong · Liang Pan · Zhongang Cai · Ziwei Liu -
2021 Poster: Unsupervised Object-Level Representation Learning from Scene Images »
Jiahao Xie · Xiaohang Zhan · Ziwei Liu · Yew Soon Ong · Chen Change Loy -
2019 : Poster and Coffee Break 1 »
Aaron Sidford · Aditya Mahajan · Alejandro Ribeiro · Alex Lewandowski · Ali H Sayed · Ambuj Tewari · Angelika Steger · Anima Anandkumar · Asier Mujika · Hilbert J Kappen · Bolei Zhou · Byron Boots · Chelsea Finn · Chen-Yu Wei · Chi Jin · Ching-An Cheng · Christina Yu · Clement Gehring · Craig Boutilier · Dahua Lin · Daniel McNamee · Daniel Russo · David Brandfonbrener · Denny Zhou · Devesh Jha · Diego Romeres · Doina Precup · Dominik Thalmeier · Eduard Gorbunov · Elad Hazan · Elena Smirnova · Elvis Dohmatob · Emma Brunskill · Enrique Munoz de Cote · Ethan Waldie · Florian Meier · Florian Schaefer · Ge Liu · Gergely Neu · Haim Kaplan · Hao Sun · Hengshuai Yao · Jalaj Bhandari · James A Preiss · Jayakumar Subramanian · Jiajin Li · Jieping Ye · Jimmy Smith · Joan Bas Serrano · Joan Bruna · John Langford · Jonathan Lee · Jose A. Arjona-Medina · Kaiqing Zhang · Karan Singh · Yuping Luo · Zafarali Ahmed · Zaiwei Chen · Zhaoran Wang · Zhizhong Li · Zhuoran Yang · Ziping Xu · Ziyang Tang · Yi Mao · David Brandfonbrener · Shirli Di-Castro · Riashat Islam · Zuyue Fu · Abhishek Naik · Saurabh Kumar · Benjamin Petit · Angeliki Kamoutsi · Simone Totaro · Arvind Raghunathan · Rui Wu · Donghwan Lee · Dongsheng Ding · Alec Koppel · Hao Sun · Christian Tjandraatmadja · Mahdi Karami · Jincheng Mei · Chenjun Xiao · Junfeng Wen · Zichen Zhang · Ross Goroshin · Mohammad Pezeshki · Jiaqi Zhai · Philip Amortila · Shuo Huang · Mariya Vasileva · El houcine Bergou · Adel Ahmadyan · Haoran Sun · Sheng Zhang · Lukas Gruber · Yuanhao Wang · Tetiana Parshakova -
2019 Poster: Transferable Normalization: Towards Improving Transferability of Deep Neural Networks »
Ximei Wang · Ying Jin · Mingsheng Long · Jianmin Wang · Michael Jordan -
2019 Poster: Policy Continuation with Hindsight Inverse Dynamics »
Hao Sun · Zhizhong Li · Xiaotong Liu · Bolei Zhou · Dahua Lin -
2019 Spotlight: Policy Continuation with Hindsight Inverse Dynamics »
Hao Sun · Zhizhong Li · Xiaotong Liu · Bolei Zhou · Dahua Lin -
2018 Poster: A Neural Compositional Paradigm for Image Captioning »
Bo Dai · Sanja Fidler · Dahua Lin -
2018 Poster: Trajectory Convolution for Action Recognition »
Yue Zhao · Yuanjun Xiong · Dahua Lin -
2017 Poster: Contrastive Learning for Image Captioning »
Bo Dai · Dahua Lin -
2013 Poster: Online Learning of Nonparametric Mixture Models via Sequential Variational Approximation »
Dahua Lin -
2012 Poster: Coupling Nonparametric Mixtures via Latent Dirichlet Processes »
Dahua Lin · John Fisher III -
2010 Oral: Construction of Dependent Dirichlet Processes based on Poisson Processes »
Dahua Lin · Eric Grimson · John Fisher III -
2010 Poster: Construction of Dependent Dirichlet Processes based on Poisson Processes »
Dahua Lin · Eric Grimson · John Fisher III