Timezone: »
The recent advances in image generation have been achieved by style-based image generators. Such approaches learn to disentangle latent factors in different image scales and encode latent factors as “style” to control image synthesis. However, existing approaches cannot further disentangle fine-grained semantics from each other, which are often conveyed from feature channels. In this paper, we propose a novel image synthesis approach by learning Semantic-aware relative importance for feature channels in Generative Adversarial Networks (SariGAN). Such a model disentangles latent factors according to the semantic of feature channels by channel-/group- wise fusion of latent codes and feature channels. Particularly, we learn to cluster feature channels by semantics and propose an adaptive group-wise Normalization (AdaGN) to independently control the styles of different channel groups. For example, we can adjust the statistics of channel groups for a human face to control the open and close of the mouth, while keeping other facial features unchanged. We propose to use adversarial training, a channel grouping loss, and a mutual information loss for joint optimization, which not only enables high-fidelity image synthesis but leads to superior interpretable properties. Extensive experiments show that our approach outperforms the SOTA style-based approaches in both unconditional image generation and conditional image inpainting tasks.
Author Information
Heliang Zheng (University of Science and Technology of China)
Jianlong Fu (Microsoft Research)
zengyh Zeng (Sun Yat-sen University)
Jiebo Luo (U. Rochester)
Zheng-Jun Zha (University of Science and Technology of China)
Related Events (a corresponding poster, oral, or spotlight)
-
2020 Poster: Learning Semantic-aware Normalization for Generative Adversarial Networks »
Tue Dec 8th 05:00 -- 07:00 PM Room Poster Session 1
More from the Same Authors
-
2021 Poster: Low-Rank Subspaces in GANs »
Jiapeng Zhu · Ruili Feng · Yujun Shen · Deli Zhao · Zheng-Jun Zha · Jingren Zhou · Qifeng Chen -
2021 Poster: Searching the Search Space of Vision Transformer »
Minghao Chen · Kan Wu · Bolin Ni · Houwen Peng · Bei Liu · Jianlong Fu · Hongyang Chao · Haibin Ling -
2021 Poster: Probing Inter-modality: Visual Parsing with Self-Attention for Vision-and-Language Pre-training »
Hongwei Xue · Yupan Huang · Bei Liu · Houwen Peng · Jianlong Fu · Houqiang Li · Jiebo Luo -
2021 Poster: Improving Visual Quality of Image Synthesis by A Token-based Generator with Transformers »
Yanhong Zeng · Huan Yang · Jianbo Wang · Hongyang Chao · Jianlong Fu -
2021 Poster: Multi-modal Dependency Tree for Video Captioning »
Wentian Zhao · Xinxiao Wu · Jiebo Luo -
2020 Poster: Cream of the Crop: Distilling Prioritized Paths For One-Shot Neural Architecture Search »
Houwen Peng · Hao Du · Hongyuan Yu · QI LI · Jing Liao · Jianlong Fu -
2020 Poster: Hierarchical Granularity Transfer Learning »
Shaobo Min · Hongtao Xie · Hantao Yao · Xuran Deng · Zheng-Jun Zha · Yongdong Zhang -
2019 Poster: Learning Deep Bilinear Transformation for Fine-grained Image Representation »
Heliang Zheng · Jianlong Fu · Zheng-Jun Zha · Jiebo Luo -
2019 Poster: Abstract Reasoning with Distracting Features »
Kecheng Zheng · Zheng-Jun Zha · Wei Wei