Timezone: »

 
Spotlight
Feature-Proxy Transformer for Few-Shot Segmentation
Jian-Wei Zhang · Yifan Sun · Yi Yang · Wei Chen

Thu Dec 08 05:00 PM -- 07:00 PM (PST) @
Few-shot segmentation~(FSS) aims at performing semantic segmentation on novel classes given a few annotated support samples. With a rethink of recent advances, we find that the current FSS framework has deviated far from the supervised segmentation framework: Given the deep features, FSS methods typically use an intricate decoder to perform sophisticated pixel-wise matching, while the supervised segmentation methods use a simple linear classification head. Due to the intricacy of the decoder and its matching pipeline, it is not easy to follow such an FSS framework. This paper revives the straightforward framework of ``feature extractor $+$ linear classification head'' and proposes a novel Feature-Proxy Transformer (FPTrans) method, in which the ``proxy'' is the vector representing a semantic class in the linear classification head. FPTrans has two keypoints for learning discriminative features and representative proxies: 1) To better utilize the limited support samples, the feature extractor makes the query interact with the support features from bottom to top layers using a novel prompting strategy. 2) FPTrans uses multiple local background proxies (instead of a single one) because the background is not homogeneous and may contain some novel foreground regions. These two keypoints are easily integrated into the vision transformer backbone with the prompting mechanism in the transformer. Given the learned features and proxies, FPTrans directly compares their cosine similarity for segmentation. Although the framework is straightforward, we show that FPTrans achieves competitive FSS accuracy on par with state-of-the-art decoder-based methods.

Author Information

Jian-Wei Zhang
Yifan Sun (Megvii Technology Inc.)
Yi Yang (Zhejiang University)
Wei Chen (State key laboratory of CAD&CG)

Related Events (a corresponding poster, oral, or spotlight)

More from the Same Authors

  • 2022 Spotlight: Lightning Talks 6B-3 »
    Lingfeng Yang · Yao Lai · Zizheng Pan · Zhenyu Wang · Weicong Liang · Chuanyang Zheng · Jian-Wei Zhang · Peng Jin · Jing Liu · Xiuying Wei · Yao Mu · Xiang Li · YUHUI YUAN · Zizheng Pan · Yifan Sun · Yunchen Zhang · Jianfei Cai · Hao Luo · zheyang li · Jinfa Huang · Haoyu He · Yi Yang · Ping Luo · Fenglin Liu · Henghui Ding · Borui Zhao · Xiangguo Zhang · Kai Zhang · Pichao WANG · Bohan Zhuang · Wei Chen · Ruihao Gong · Zhi Yang · Xian Wu · Feng Ding · Jianfei Cai · Xiao Luo · Renjie Song · Weihong Lin · Jian Yang · Wenming Tan · Bohan Zhuang · Shanghang Zhang · Shen Ge · Fan Wang · Qi Zhang · Guoli Song · Jun Xiao · Hao Li · Ding Jia · David Clifton · Ye Ren · Fengwei Yu · Zheng Zhang · Jie Chen · Shiliang Pu · Xianglong Liu · Chao Zhang · Han Hu
  • 2022 Spotlight: Decoupling Features in Hierarchical Propagation for Video Object Segmentation »
    Zongxin Yang · Yi Yang
  • 2022 Spotlight: Lightning Talks 6A-1 »
    Ziyi Wang · Nian Liu · Yaming Yang · Qilong Wang · Yuanxin Liu · Zongxin Yang · Yizhao Gao · Yanchen Deng · Dongze Lian · Nanyi Fei · Ziyu Guan · Xiao Wang · Shufeng Kong · Xumin Yu · Daquan Zhou · Yi Yang · Fandong Meng · Mingze Gao · Caihua Liu · Yongming Rao · Zheng Lin · Haoyu Lu · Zhe Wang · Jiashi Feng · Zhaolin Zhang · Deyu Bo · Xinchao Wang · Chuan Shi · Jiangnan Li · Jiangtao Xie · Jie Zhou · Zhiwu Lu · Wei Zhao · Bo An · Jiwen Lu · Peihua Li · Jian Pei · Hao Jiang · Cai Xu · Peng Fu · Qinghua Hu · Yijie Li · Weigang Lu · Yanan Cao · Jianbin Huang · Weiping Wang · Zhao Cao · Jie Zhou
  • 2022 Spotlight: Lightning Talks 3B-4 »
    Guanghu Yuan · Yijing Liu · Li Yang · Yongri Piao · Zekang Zhang · Yaxin Xiao · Lin Chen · Hong Chang · Fajie Yuan · Guangyu Gao · Hong Chang · Qinxian Liu · Zhixiang Wei · Qingqing Ye · Chenyang Lu · Jian Meng · Haibo Hu · Xin Jin · Yudong Li · Miao Zhang · Zhiyuan Fang · Jae-sun Seo · Bingpeng MA · Jian-Wei Zhang · Shiguang Shan · Haozhe Feng · Huaian Chen · Deliang Fan · Huadi Zheng · Jianbo Jiao · Huchuan Lu · Beibei Kong · Miao Zheng · Chengfang Fang · Shujie Li · Zhongwei Wang · Yunchao Wei · Xilin Chen · Jie Shi · Kai Chen · Zihan Zhou · Lei Chen · Yi Jin · Wei Chen · Min Yang · Chenyun YU · Bo Hu · Zang Li · Yu Xu · Xiaohu Qie
  • 2022 Spotlight: Multivariate Time-Series Forecasting with Temporal Polynomial Graph Neural Networks »
    Yijing Liu · Qinxian Liu · Jian-Wei Zhang · Haozhe Feng · Zhongwei Wang · Zihan Zhou · Wei Chen
  • 2022 Spotlight: Lightning Talks 1A-4 »
    Siwei Wang · Jing Liu · Nianqiao Ju · Shiqian Li · Elo├»se Berthier · Muhammad Faaiz Taufiq · Arsene Fansi Tchango · Chen Liang · Chulin Xie · Jordan Awan · Jean-Francois Ton · Ziad Kobeissi · Wenguan Wang · Xinwang Liu · Kewen Wu · Rishab Goel · Jiaxu Miao · Suyuan Liu · Julien Martel · Ruobin Gong · Francis Bach · Chi Zhang · Rob Cornish · Sanmi Koyejo · Zhi Wen · Yee Whye Teh · Yi Yang · Jiaqi Jin · Bo Li · Yixin Zhu · Vinayak Rao · Wenxuan Tu · Gaetan Marceau Caron · Arnaud Doucet · Xinzhong Zhu · Joumana Ghosn · En Zhu
  • 2022 Spotlight: GMMSeg: Gaussian Mixture based Generative Semantic Segmentation Models »
    Chen Liang · Wenguan Wang · Jiaxu Miao · Yi Yang
  • 2022 Poster: GMMSeg: Gaussian Mixture based Generative Semantic Segmentation Models »
    Chen Liang · Wenguan Wang · Jiaxu Miao · Yi Yang
  • 2022 Poster: Multivariate Time-Series Forecasting with Temporal Polynomial Graph Neural Networks »
    Yijing Liu · Qinxian Liu · Jian-Wei Zhang · Haozhe Feng · Zhongwei Wang · Zihan Zhou · Wei Chen
  • 2022 Poster: Decoupling Features in Hierarchical Propagation for Video Object Segmentation »
    Zongxin Yang · Yi Yang
  • 2021 Poster: Spatial Ensemble: a Novel Model Smoothing Mechanism for Student-Teacher Framework »
    Tengteng Huang · Yifan Sun · Xun Wang · Haotian Yao · Chi Zhang