Timezone: »

 
Spotlight
AniFaceGAN: Animatable 3D-Aware Face Image Generation for Video Avatars
Yue Wu · Yu Deng · Jiaolong Yang · Fangyun Wei · Qifeng Chen · Xin Tong

Wed Dec 07 05:00 PM -- 07:00 PM (PST) @
in Lightning Talks 4B-3 »

Although 2D generative models have made great progress in face image generation and animation, they often suffer from undesirable artifacts such as 3D inconsistency when rendering images from different camera viewpoints. This prevents them from synthesizing video animations indistinguishable from real ones. Recently, 3D-aware GANs extend 2D GANs for explicit disentanglement of camera pose by leveraging 3D scene representations. These methods can well preserve the 3D consistency of the generated images across different views, yet they cannot achieve fine-grained control over other attributes, among which facial expression control is arguably the most useful and desirable for face animation. In this paper, we propose an animatable 3D-aware GAN for multiview consistent face animation generation. The key idea is to decompose the 3D representation of the 3D-aware GAN into a template field and a deformation field, where the former represents different identities with a canonical expression, and the latter characterizes expression variations of each identity. To achieve meaningful control over facial expressions via deformation, we propose a 3D-level imitative learning scheme between the generator and a parametric 3D face model during adversarial training of the 3D-aware GAN. This helps our method achieve high-quality animatable face image generation with strong visual 3D consistency, even though trained with only unstructured 2D images. Extensive experiments demonstrate our superior performance over prior works. Project page: \url{https://yuewuhkust.github.io/AniFaceGAN/

Author Information

Yue Wu (HKUST)
Yu Deng (Xiaobing.AI)
Jiaolong Yang (Microsoft Research)
Fangyun Wei (Microsoft Research Asia)
Qifeng Chen (Hong Kong University of Science and Technology)
Xin Tong (Microsoft Research Asia)

Related Events (a corresponding poster, oral, or spotlight)

More from the Same Authors