Timezone: »
Recent studies have shown that the performance of forgery detection can be improved with diverse and challenging Deepfakes datasets. However, due to the lack of Deepfakes datasets with large variance in appearance, which can be hardly produced by recent identity swapping methods, the detection algorithm may fail in this situation. In this work, we provide a new identity swapping algorithm with large differences in appearance for face forgery detection. The appearance gaps mainly arise from the large discrepancies in illuminations and skin colors that widely exist in real-world scenarios. However, due to the difficulties of modeling the complex appearance mapping, it is challenging to transfer fine-grained appearances adaptively while preserving identity traits. This paper formulates appearance mapping as an optimal transport problem and proposes an Appearance Optimal Transport model (AOT) to formulate it in both latent and pixel space. Specifically, a relighting generator is designed to simulate the optimal transport plan. It is solved via minimizing the Wasserstein distance of the learned features in the latent space, enabling better performance and less computation than conventional optimization. To further refine the solution of the optimal transport plan, we develop a segmentation game to minimize the Wasserstein distance in the pixel space. A discriminator is introduced to distinguish the fake parts from a mix of real and fake image patches. Extensive experiments reveal that the superiority of our method when compared with state-of-the-art methods and the ability of our generated data to improve the performance of face forgery detection.
Author Information
Hao Zhu (Anhui University)
Chaoyou Fu (Institute of Automation, Chinese Academy of Sciences)
Qianyi Wu (Sensetime)
Wayne Wu (Tsinghua University)
Chen Qian (SenseTime)
Ran He (NLPR, CASIA)
More from the Same Authors
-
2022 Poster: Are You Stealing My Model? Sample Correlation for Fingerprinting Deep Neural Networks »
Jiyang Guan · Jian Liang · Ran He -
2022 Poster: Weak-shot Semantic Segmentation via Dual Similarity Transfer »
Junjie Chen · Li Niu · Siyuan Zhou · Jianlou Si · Chen Qian · Liqing Zhang -
2022 Spotlight: Lightning Talks 6B-4 »
Junjie Chen · Chuanxia Zheng · JINLONG LI · Yu Shi · Shichao Kan · Yu Wang · FermÃn Travi · Ninh Pham · Lei Chai · Guobing Gan · Tung-Long Vuong · Gonzalo Ruarte · Tao Liu · Li Niu · Jingjing Zou · Zequn Jie · Peng Zhang · Ming LI · Yixiong Liang · Guolin Ke · Jianfei Cai · Gaston Bujia · Sunzhu Li · Siyuan Zhou · Jingyang Lin · Xu Wang · Min Li · Zhuoming Chen · Qing Ling · Xiaolin Wei · Xiuqing Lu · Shuxin Zheng · Dinh Phung · Yigang Cen · Jianlou Si · Juan Esteban Kamienkowski · Jianxin Wang · Chen Qian · Lin Ma · Benyou Wang · Yingwei Pan · Tie-Yan Liu · Liqing Zhang · Zhihai He · Ting Yao · Tao Mei -
2022 Spotlight: Weak-shot Semantic Segmentation via Dual Similarity Transfer »
Junjie Chen · Li Niu · Siyuan Zhou · Jianlou Si · Chen Qian · Liqing Zhang -
2022 Spotlight: Are You Stealing My Model? Sample Correlation for Fingerprinting Deep Neural Networks »
Jiyang Guan · Jian Liang · Ran He -
2022 Spotlight: Lightning Talks 3A-1 »
Shu Ding · Wanxing Chang · Jiyang Guan · Mouxiang Chen · Guan Gui · Yue Tan · Shiyun Lin · Guodong Long · Yuze Han · Wei Wang · Zhen Zhao · Ye Shi · Jian Liang · Chenghao Liu · Lei Qi · Ran He · Jie Ma · Zemin Liu · Xiang Li · Hoang Tuan · Luping Zhou · Zhihua Zhang · Jianling Sun · Jingya Wang · LU LIU · Tianyi Zhou · Lei Wang · Jing Jiang · Yinghuan Shi -
2022 Poster: Knowledge Distillation from A Stronger Teacher »
Tao Huang · Shan You · Fei Wang · Chen Qian · Chang Xu -
2022 Poster: Green Hierarchical Vision Transformer for Masked Image Modeling »
Lang Huang · Shan You · Mingkai Zheng · Fei Wang · Chen Qian · Toshihiko Yamasaki -
2022 Poster: Orthogonal Transformer: An Efficient Vision Transformer Backbone with Token Orthogonalization »
Huaibo Huang · Xiaoqiang Zhou · Ran He -
2021 Poster: Deceive D: Adaptive Pseudo Augmentation for GAN Training with Limited Data »
Liming Jiang · Bo Dai · Wayne Wu · Chen Change Loy -
2021 Poster: ReSSL: Relational Self-Supervised Learning with Weak Augmentation »
Mingkai Zheng · Shan You · Fei Wang · Chen Qian · Changshui Zhang · Xiaogang Wang · Chang Xu -
2020 Poster: Agree to Disagree: Adaptive Ensemble Knowledge Distillation in Gradient Space »
Shangchen Du · Shan You · Xiaojie Li · Jianlong Wu · Fei Wang · Chen Qian · Changshui Zhang -
2020 Poster: ISTA-NAS: Efficient and Consistent Neural Architecture Search by Sparse Coding »
Yibo Yang · Hongyang Li · Shan You · Fei Wang · Chen Qian · Zhouchen Lin -
2019 Poster: Dual Variational Generation for Low Shot Heterogeneous Face Recognition »
Chaoyou Fu · Xiang Wu · Yibo Hu · Huaibo Huang · Ran He -
2019 Spotlight: Dual Variational Generation for Low Shot Heterogeneous Face Recognition »
Chaoyou Fu · Xiang Wu · Yibo Hu · Huaibo Huang · Ran He -
2018 Poster: Learning a High Fidelity Pose Invariant Model for High-resolution Face Frontalization »
Jie Cao · Yibo Hu · Hongwen Zhang · Ran He · Zhenan Sun -
2018 Poster: IntroVAE: Introspective Variational Autoencoders for Photographic Image Synthesis »
Huaibo Huang · zhihang li · Ran He · Zhenan Sun · Tieniu Tan -
2017 Poster: Deep Supervised Discrete Hashing »
Qi Li · Zhenan Sun · Ran He · Tieniu Tan