Timezone: »

CATs: Cost Aggregation Transformers for Visual Correspondence
Seokju Cho · Sunghwan Hong · Sangryul Jeon · Yunsung Lee · Kwanghoon Sohn · Seungryong Kim

Thu Dec 09 12:30 AM -- 02:00 AM (PST) @

We propose a novel cost aggregation network, called Cost Aggregation Transformers (CATs), to find dense correspondences between semantically similar images with additional challenges posed by large intra-class appearance and geometric variations. Cost aggregation is a highly important process in matching tasks, which the matching accuracy depends on the quality of its output. Compared to hand-crafted or CNN-based methods addressing the cost aggregation, in that either lacks robustness to severe deformations or inherit the limitation of CNNs that fail to discriminate incorrect matches due to limited receptive fields, CATs explore global consensus among initial correlation map with the help of some architectural designs that allow us to fully leverage self-attention mechanism. Specifically, we include appearance affinity modeling to aid the cost aggregation process in order to disambiguate the noisy initial correlation maps and propose multi-level aggregation to efficiently capture different semantics from hierarchical feature representations. We then combine with swapping self-attention technique and residual connections not only to enforce consistent matching, but also to ease the learning process, which we find that these result in an apparent performance boost. We conduct experiments to demonstrate the effectiveness of the proposed model over the latest methods and provide extensive ablation studies. Code and trained models are available at https://sunghwanhong.github.io/CATs/.

Author Information

Seokju Cho (Yonsei University)
Sunghwan Hong (Korea University)
Sangryul Jeon (Yonsei University)
Yunsung Lee (Korea University)
Kwanghoon Sohn (Yonsei Univ.)
Seungryong Kim (Korea University)

More from the Same Authors

  • 2023 Poster: DäRF: Boosting Few-shot Neural Radiance Field with Joint Monocular Depth Adaptation »
    Jiuhn Song · Seonghoon Park · Honggyu An · Seokju Cho · Min-Seop Kwak · Sungjin Cho · Seungryong Kim
  • 2023 Poster: Debiasing Scores and Prompts of 2D Diffusion for View-consistent Text-to-3D Generation »
    Susung Hong · Donghoon Ahn · Seungryong Kim
  • 2022 Poster: Neural Matching Fields: Implicit Representation of Matching Fields for Visual Correspondence »
    Sunghwan Hong · Jisu Nam · Seokju Cho · Susung Hong · Sangryul Jeon · Dongbo Min · Seungryong Kim
  • 2021 Poster: SWAD: Domain Generalization by Seeking Flat Minima »
    Junbum Cha · Sanghyuk Chun · Kyungjae Lee · Han-Cheol Cho · Seunghyun Park · Yunsung Lee · Sungrae Park
  • 2019 : Lunch + Poster Session »
    Frederik Gerzer · Bill Yang Cai · Pieter-Jan Hoedt · Kelly Kochanski · Soo Kyung Kim · Yunsung Lee · Sunghyun Park · Sharon Zhou · Martin Gauch · Jonathan Wilson · Joyjit Chatterjee · Shamindra Shrotriya · Dimitri Papadimitriou · Christian Schön · Valentina Zantedeschi · Gabriella Baasch · Willem Waegeman · Gautier Cosne · Dara Farrell · Brendan Lucier · Letif Mones · Caleb Robinson · Tafara Chitsiga · Victor Kristof · Hari Prasanna Das · Yimeng Min · Alexandra Puchko · Alexandra Luccioni · Kyle Story · Jason Hickey · Yue Hu · Björn Lütjens · Zhecheng Wang · Renzhi Jing · Genevieve Flaspohler · Jingfan Wang · Saumya Sinha · Qinghu Tang · Armi Tiihonen · Ruben Glatt · Muge Komurcu · Jan Drgona · Juan Gomez-Romero · Ashish Kapoor · Dylan J Fitzpatrick · Alireza Rezvanifar · Adrian Albert · Olya (Olga) Irzak · Kara Lamb · Ankur Mahesh · Kiwan Maeng · Frederik Kratzert · Sorelle Friedler · Niccolo Dalmasso · Alex Robson · Lindiwe Malobola · Lucas Maystre · Yu-wen Lin · Surya Karthik Mukkavili · Brian Hutchinson · Alexandre Lacoste · Yanbing Wang · Zhengcheng Wang · Yinda Zhang · Victoria Preston · Jacob Pettit · Draguna Vrabie · Miguel Molina-Solana · Tonio Buonassisi · Andrew Annex · Tunai P Marques · Catalin Voss · Johannes Rausch · Max Evans
  • 2018 Poster: Recurrent Transformer Networks for Semantic Correspondence »
    Seungryong Kim · Stephen Lin · Sangryul Jeon · Dongbo Min · Kwanghoon Sohn
  • 2018 Spotlight: Recurrent Transformer Networks for Semantic Correspondence »
    Seungryong Kim · Stephen Lin · Sangryul Jeon · Dongbo Min · Kwanghoon Sohn