Timezone: »
In this work, we focus on the task of learning and representing dense correspondences in deformable object categories. While this problem has been considered before, solutions so far have been rather ad-hoc for specific object types (i.e., humans), often with significant manual work involved. However, scaling the geometry understanding to all objects in nature requires more automated approaches that can also express correspondences between related, but geometrically different objects. To this end, we propose a new, learnable image-based representation of dense correspondences. Our model predicts, for each pixel in a 2D image, an embedding vector of the corresponding vertex in the object mesh, therefore establishing dense correspondences between image pixels and 3D object geometry. We demonstrate that the proposed approach performs on par or better than the state-of-the-art methods for dense pose estimation for humans, while being conceptually simpler. We also collect a new in-the-wild dataset of dense correspondences for animal classes and demonstrate that our framework scales naturally to the new deformable object categories.
Author Information
Natalia Neverova (Facebook AI Research)
David Novotny (Facebook AI Research)
Marc Szafraniec (Facebook AI Research)
Vasil Khalidov (Facebook AI Research)
Patrick Labatut (Facebook AI Research)
Andrea Vedaldi (University of Oxford / Facebook AI Research)
More from the Same Authors
-
2021 : PASS: An ImageNet replacement for self-supervised pretraining without humans »
Yuki Asano · Christian Rupprecht · Andrew Zisserman · Andrea Vedaldi -
2021 : PASS: An ImageNet replacement for self-supervised pretraining without humans »
Yuki Asano · Christian Rupprecht · Andrew Zisserman · Andrea Vedaldi -
2021 Poster: Keeping Your Eye on the Ball: Trajectory Attention in Video Transformers »
Mandela Patrick · Dylan Campbell · Yuki Asano · Ishan Misra · Florian Metze · Christoph Feichtenhofer · Andrea Vedaldi · João Henriques -
2021 Poster: XCiT: Cross-Covariance Image Transformers »
Alaaeldin Ali · Hugo Touvron · Mathilde Caron · Piotr Bojanowski · Matthijs Douze · Armand Joulin · Ivan Laptev · Natalia Neverova · Gabriel Synnaeve · Jakob Verbeek · Herve Jegou -
2021 Oral: Keeping Your Eye on the Ball: Trajectory Attention in Video Transformers »
Mandela Patrick · Dylan Campbell · Yuki Asano · Ishan Misra · Florian Metze · Christoph Feichtenhofer · Andrea Vedaldi · João Henriques -
2021 Poster: Unsupervised Part Discovery from Contrastive Reconstruction »
Subhabrata Choudhury · Iro Laina · Christian Rupprecht · Andrea Vedaldi -
2021 Poster: DOBF: A Deobfuscation Pre-Training Objective for Programming Languages »
Marie-Anne Lachaux · Baptiste Roziere · Marc Szafraniec · Guillaume Lample -
2020 Poster: Labelling unlabelled videos from scratch with multi-modal self-supervision »
Yuki Asano · Mandela Patrick · Christian Rupprecht · Andrea Vedaldi -
2020 Poster: Canonical 3D Deformer Maps: Unifying parametric and non-parametric methods for dense weakly-supervised category reconstruction »
David Novotny · Roman Shapovalov · Andrea Vedaldi -
2020 Poster: 3D Multi-bodies: Fitting Sets of Plausible 3D Human Models to Ambiguous Image Data »
Benjamin Biggs · David Novotny · Sebastien Ehrhardt · Hanbyul Joo · Ben Graham · Andrea Vedaldi -
2020 Spotlight: 3D Multi-bodies: Fitting Sets of Plausible 3D Human Models to Ambiguous Image Data »
Benjamin Biggs · David Novotny · Sebastien Ehrhardt · Hanbyul Joo · Ben Graham · Andrea Vedaldi -
2020 Session: Orals & Spotlights Track 07: Vision Applications »
Ce Liu · Natalia Neverova -
2019 : Carl Doersch, Raquel Urtasun, Sanja Fidler moderated by Natalia Neverova »
Raquel Urtasun · Sanja Fidler · Natalia Neverova · Ilija Radosavovic · Carl Doersch -
2019 Poster: PerspectiveNet: A Scene-consistent Image Generator for New View Synthesis in Real Indoor Environments »
David Novotny · Ben Graham · Jeremy Reizenstein -
2019 Poster: Correlated Uncertainty for Learning Dense Correspondences from Noisy Labels »
Natalia Neverova · David Novotny · Andrea Vedaldi -
2018 Poster: Forward Modeling for Partial Observation Strategy Games - A StarCraft Defogger »
Gabriel Synnaeve · Zeming Lin · Jonas Gehring · Dan Gant · Vegard Mella · Vasil Khalidov · Nicolas Carion · Nicolas Usunier -
2017 Poster: Houdini: Fooling Deep Structured Visual and Speech Recognition Models with Adversarial Examples »
Moustapha Cisse · Yossi Adi · Natalia Neverova · Joseph Keshet