Timezone: »
The latent space of a Generative Adversarial Network (GAN) has been shown to encode rich semantics within some subspaces. To identify these subspaces, researchers typically analyze the statistical information from a collection of synthesized data, and the identified subspaces tend to control image attributes globally (i.e., manipulating an attribute causes the change of an entire image). By contrast, this work introduces low-rank subspaces that enable more precise control of GAN generation. Concretely, given an arbitrary image and a region of interest (e.g., eyes of face images), we manage to relate the latent space to the image region with the Jacobian matrix and then use low-rank factorization to discover steerable latent subspaces. There are three distinguishable strengths of our approach that can be aptly called LowRankGAN. First, compared to analytic algorithms in prior work, our low-rank factorization of Jacobians is able to find the low-dimensional representation of attribute manifold, making image editing more precise and controllable. Second, low-rank factorization naturally yields a null space of attributes such that moving the latent code within it only affects the outer region of interest. Therefore, local image editing can be simply achieved by projecting an attribute vector into the null space without relying on a spatial mask as existing methods do. Third, our method can robustly work with a local region from one image for analysis yet well generalize to other images, making it much easy to use in practice. Extensive experiments on state-of-the-art GAN models (including StyleGAN2 and BigGAN) trained on various datasets demonstrate the effectiveness of our LowRankGAN.
Author Information
Jiapeng Zhu (Hong Kong University of Science and Technology)
Ruili Feng (University of Science and Technology of China)
Yujun Shen (ByteDance Inc.)
Deli Zhao (Xiaomi AI Lab)
Zheng-Jun Zha (University of Science and Technology of China)
Jingren Zhou (Alibaba Group)
Qifeng Chen (HKUST)
More from the Same Authors
-
2021 Poster: Data-Efficient Instance Generation from Instance Discrimination »
Ceyuan Yang · Yujun Shen · Yinghao Xu · Bolei Zhou -
2021 Poster: UFC-BERT: Unifying Multi-Modal Controls for Conditional Image Synthesis »
Zhu Zhang · Jianxin Ma · Chang Zhou · Rui Men · Zhikang Li · Ming Ding · Jie Tang · Jingren Zhou · Hongxia Yang -
2020 Poster: Hierarchical Granularity Transfer Learning »
Shaobo Min · Hongtao Xie · Hantao Yao · Xuran Deng · Zheng-Jun Zha · Yongdong Zhang -
2020 Poster: Blind Video Temporal Consistency via Deep Video Prior »
Chenyang Lei · Yazhou Xing · Qifeng Chen -
2020 Poster: Learning to Mutate with Hypergradient Guided Population »
Zhiqiang Tao · Yaliang Li · Bolin Ding · Ce Zhang · Jingren Zhou · Yun Fu -
2020 Poster: Learning Semantic-aware Normalization for Generative Adversarial Networks »
Heliang Zheng · Jianlong Fu · Yanhong Zeng · Jiebo Luo · Zheng-Jun Zha -
2020 Spotlight: Learning Semantic-aware Normalization for Generative Adversarial Networks »
Heliang Zheng · Jianlong Fu · Yanhong Zeng · Jiebo Luo · Zheng-Jun Zha -
2019 Poster: Learning Deep Bilinear Transformation for Fine-grained Image Representation »
Heliang Zheng · Jianlong Fu · Zheng-Jun Zha · Jiebo Luo -
2019 Poster: Abstract Reasoning with Distracting Features »
Kecheng Zheng · Zheng-Jun Zha · Wei Wei -
2018 Poster: Combinatorial Optimization with Graph Convolutional Networks and Guided Tree Search »
Zhuwen Li · Qifeng Chen · Vladlen Koltun -
2018 Poster: DeepExposure: Learning to Expose Photos with Asynchronously Reinforced Adversarial Learning »
Runsheng Yu · Wenyu Liu · Yasen Zhang · Zhi Qu · Deli Zhao · Bo Zhang -
2014 Poster: Zeta Hull Pursuits: Learning Nonconvex Data Hulls »
Yuanjun Xiong · Wei Liu · Deli Zhao · Xiaoou Tang -
2008 Poster: Cyclizing Clusters via Zeta Function of a Graph »
Deli Zhao · Xiaoou Tang