Timezone: »

Temporal Transductive Inference for Few-Shot Video Object Segmentation
Mennatullah Siam · Richard Wildes

Few-shot object segmentation has been focused on segmenting static images in the query set. Recently few-shot video object segmentation (FS-VOS), where the query images to be segmented belong to a video, has been introduced but is still under-explored. We propose a simple but effective temporal transductive inference (TTI) that uses the temporal continuity in videos to improve the segmentation with a few-shot support set. We use both global and local cues. Global cues focus on learning a consistent prototype on the sequence level, whereas local cues focus on a consistent foreground/background region proportion within a local temporal window. Our model outperforms state-of-the-art attention-based counterpart on few-shot Youtube-VIS with 2% in mean intersection over union (mIoU). Finally, we propose a more realistic FS-VOS setup that operates cross-domain. Our method outperforms the transductive inference baseline that uses static images with 1.3% improvement on two different benchmarks. It demonstrates that our method is a promising direction and opens the door towards a label efficient approach of annotating video datasets with rare classes that occur in different robotics settings such as autonomous driving.

Author Information

Mennatullah Siam (University of Alberta)
Richard Wildes (York University)

More from the Same Authors

  • 2020 : Paper 7: Real-time Semantic and Class-agnostic Instance Segmentation in Autonomous Driving »
    Mennatullah Siam · Hazem Rashed · Ahmad El Sallab
  • 2018 : Poster Session »
    Carl Trimbach · Mennatullah Siam · Rodrigo Toro Icarte · Zhongtian Dai · Sheila McIlraith · Matthew Rahtz · Robert Sheline · Christopher MacLellan · Carolin Lawrence · Stefan Riezler · Dylan Hadfield-Menell · Fang-I Hsiao
  • 2017 : 6 Spotlight Talks (3 min each) »
    Mennatullah Siam · Mohit Prabhushankar · Priyam Parashar · Mustafa Mukadam · hengshuai yao · Ransalu Senanayake
  • 2017 : Posters and Coffee »
    Jean-Baptiste Tristan · Yunseong Lee · Anna Veronika Dorogush · Shohei Hido · Michael Terry · Mennatullah Siam · Hidemoto Nakada · Cody Coleman · Jung-Woo Ha · Hao Zhang · Adam Stooke · Chen Meng · Christopher Kappler · Lane Schwartz · Christopher Olston · Sebastian Schelter · Minmin Sun · Daniel Kang · Waldemar Hummer · Jichan Chung · Tim Kraska · Kannan Ramchandran · Nick Hynes · Christoph Boden · Donghyun Kwak
  • 2016 Poster: Spatiotemporal Residual Networks for Video Action Recognition »
    Christoph Feichtenhofer · Axel Pinz · Richard Wildes