Timezone: »
Image animation consists of generating a video sequence so that an object in a source image is animated according to the motion of a driving video. Our framework addresses this problem without using any annotation or prior information about the specific object to animate. Once trained on a set of videos depicting objects of the same category (e.g. faces, human bodies), our method can be applied to any object of this class. To achieve this, we decouple appearance and motion information using a self-supervised formulation. To support complex motions, we use a representation consisting of a set of learned keypoints along with their local affine transformations. A generator network models occlusions arising during target motions and combines the appearance extracted from the source image and the motion derived from the driving video. Our framework scores best on diverse benchmarks and on a variety of object categories.
Author Information
Aliaksandr Siarohin (University of Trento)
Stéphane Lathuilière (Telecom-Paris)
Sergey Tulyakov (Snap Inc)
Elisa Ricci (FBK - Technologies of Vision)
Nicu Sebe (University of Trento)
More from the Same Authors
-
2022 Poster: RankFeat: Rank-1 Feature Removal for Out-of-distribution Detection »
Yue Song · Nicu Sebe · Wei Wang -
2022 Spotlight: EpiGRAF: Rethinking training of 3D GANs »
Ivan Skorokhodov · Sergey Tulyakov · Yiqun Wang · Peter Wonka -
2022 Spotlight: Lightning Talks 5B-1 »
Devansh Arpit · Xiaojun Xu · Zifan Shi · Ivan Skorokhodov · Shayan Shekarforoush · Zhan Tong · Yiqun Wang · Shichong Peng · Linyi Li · Ivan Skorokhodov · Huan Wang · Yibing Song · David Lindell · Yinghao Xu · Seyed Alireza Moazenipourasil · Sergey Tulyakov · Peter Wonka · Yiqun Wang · Ke Li · David Fleet · Yujun Shen · Yingbo Zhou · Bo Li · Jue Wang · Peter Wonka · Marcus Brubaker · Caiming Xiong · Limin Wang · Deli Zhao · Qifeng Chen · Dit-Yan Yeung -
2022 Poster: EpiGRAF: Rethinking training of 3D GANs »
Ivan Skorokhodov · Sergey Tulyakov · Yiqun Wang · Peter Wonka -
2022 Poster: Adversarial Style Augmentation for Domain Generalized Urban-Scene Segmentation »
Zhun Zhong · Yuyang Zhao · Gim Hee Lee · Nicu Sebe -
2022 Poster: Layer Freezing & Data Sieving: Missing Pieces of a Generic Framework for Sparse Training »
Geng Yuan · Yanyu Li · Sheng Li · Zhenglun Kong · Sergey Tulyakov · Xulong Tang · Yanzhi Wang · Jian Ren -
2022 Poster: EfficientFormer: Vision Transformers at MobileNet Speed »
Yanyu Li · Geng Yuan · Yang Wen · Ju Hu · Georgios Evangelidis · Sergey Tulyakov · Yanzhi Wang · Jian Ren -
2021 Poster: Efficient Training of Visual Transformers with Small Datasets »
Yahui Liu · Enver Sangineto · Wei Bi · Nicu Sebe · Bruno Lepri · Marco Nadai -
2020 Poster: Latent World Models For Intrinsically Motivated Exploration »
Aleksandr Ermolov · Nicu Sebe -
2020 Spotlight: Latent World Models For Intrinsically Motivated Exploration »
Aleksandr Ermolov · Nicu Sebe -
2017 Poster: Learning Deep Structured Multi-Scale Features using Attention-Gated CRFs for Contour Prediction »
Dan Xu · Wanli Ouyang · Xavier Alameda-Pineda · Elisa Ricci · Xiaogang Wang · Nicu Sebe