Skip to yearly menu bar Skip to main content


Universal Correspondence Network

Christopher B Choy · Manmohan Chandraker · JunYoung Gwak · Silvio Savarese

Area 5+6+7+8 #185

Keywords: [ Similarity and Distance Learning ] [ (Application) Object and Pattern Recognition ] [ (Application) Computer Vision ] [ Deep Learning or Neural Networks ]

Abstract: We present a deep learning framework for accurate visual correspondences and demonstrate its effectiveness for both geometric and semantic matching, spanning across rigid motions to intra-class shape or appearance variations. In contrast to previous CNN-based approaches that optimize a surrogate patch similarity objective, we use deep metric learning to directly learn a feature space that preserves either geometric or semantic similarity. Our fully convolutional architecture, along with a novel correspondence contrastive loss allows faster training by effective reuse of computations, accurate gradient computation through the use of thousands of examples per image pair and faster testing with $O(n)$ feedforward passes for n keypoints, instead of $O(n^2)$ for typical patch similarity methods. We propose a convolutional spatial transformer to mimic patch normalization in traditional features like SIFT, which is shown to dramatically boost accuracy for semantic correspondences across intra-class shape variations. Extensive experiments on KITTI, PASCAL and CUB-2011 datasets demonstrate the significant advantages of our features over prior works that use either hand-constructed or learned features.

Live content is unavailable. Log in and register to view live content