Timezone: »

 
Poster
FNeVR: Neural Volume Rendering for Face Animation
Bohan Zeng · Boyu Liu · Hong Li · Xuhui Liu · Jianzhuang Liu · Dapeng Chen · Wei Peng · Baochang Zhang

@

Face animation, one of the hottest topics in computer vision, has achieved a promising performance with the help of generative models. However, it remains a critical challenge to generate identity preserving and photo-realistic images due to the sophisticated motion deformation and complex facial detail modeling. To address these problems, we propose a Face Neural Volume Rendering (FNeVR) network to fully explore the potential of 2D motion warping and 3D volume rendering in a unified framework. In FNeVR, we design a 3D Face Volume Rendering (FVR) module to enhance the facial details for image rendering. Specifically, we first extract 3D information with a well designed architecture, and then introduce an orthogonal adaptive ray-sampling module for efficient rendering. We also design a lightweight pose editor, enabling FNeVR to edit the facial pose in a simple yet effective way. Extensive experiments show that our FNeVR obtains the best overall quality and performance on widely used talking-head benchmarks.

Keywords: image synthesis · neural volume rendering · lightweight pose editor · face animation
[ Paper]  [ Slides]  [ Poster]  [ OpenReview] 

Author Information

Bohan Zeng (Beijing University of Aeronautics and Astronautics)
Boyu Liu (BUAA)
Hong Li (Beijing University of Aeronautics and Astronautics)
Xuhui Liu (Beihang University)
Jianzhuang Liu (Huawei Noah's Ark Lab)
Dapeng Chen (The Chinese University of Hong Kong)
Wei Peng (Huawei Technologies Ltd.)

Wei Peng is the director and principal AI scientist at the AI Application Research Center, Huawei Technologies Co., Ltd., responsible for the research and development of AI products and services. Prior to joining Huawei, Wei was a senior lecturer at La Trobe University (2016-19). He was awarded a Ph.D. at The University of Sydney, Australia, in 2007. Wei’s research interests include cognitive systems, neural machine translation, dialogue systems, and multimodal representation. Wei has published over 50 papers in peer-reviewed journals and conferences in the field of AI. He regularly serves as PC for international conferences, including ACL, EMNLP, WMT, etc.

Baochang Zhang (Beihang University)

More from the Same Authors