NIPS 2018

Timezone: »

Poster

Soft-Gated Warping-GAN for Pose-Guided Person Image Synthesis

Haoye Dong · Xiaodan Liang · Ke Gong · Hanjiang Lai · Jia Zhu · Jian Yin

Tue Dec 04 07:45 AM -- 09:45 AM (PST) @ Room 517 AB #146

in Tue Poster Session A »

Despite remarkable advances in image synthesis research, existing works often fail in manipulating images under the context of large geometric transformations. Synthesizing person images conditioned on arbitrary poses is one of the most representative examples where the generation quality largely relies on the capability of identifying and modeling arbitrary transformations on different body parts. Current generative models are often built on local convolutions and overlook the key challenges (e.g. heavy occlusions, different views or dramatic appearance changes) when distinct geometric changes happen for each part, caused by arbitrary pose manipulations. This paper aims to resolve these challenges induced by geometric variability and spatial displacements via a new Soft-Gated Warping Generative Adversarial Network (Warping-GAN), which is composed of two stages: 1) it first synthesizes a target part segmentation map given a target pose, which depicts the region-level spatial layouts for guiding image synthesis with higher-level structure constraints; 2) the Warping-GAN equipped with a soft-gated warping-block learns feature-level mapping to render textures from the original image into the generated segmentation map. Warping-GAN is capable of controlling different transformation degrees given distinct target poses. Moreover, the proposed warping-block is light-weight and flexible enough to be injected into any networks. Human perceptual studies and quantitative evaluations demonstrate the superiority of our Warping-GAN that significantly outperforms all existing methods on two large datasets.

Author Information

Haoye Dong (Sun Yat-sen University)

Xiaodan Liang (Sun Yat-sen University)

Ke Gong (Sun Yat-sen University )

Hanjiang Lai (Sun Yat-Sen university)

Jia Zhu (South China Normal University)

Jian Yin (Sun Yat-Sen University)

More from the Same Authors