Timezone: »
The latent space of a Generative Adversarial Network (GAN) has been shown to encode rich semantics within some subspaces. To identify these subspaces, researchers typically analyze the statistical information from a collection of synthesized data, and the identified subspaces tend to control image attributes globally (i.e., manipulating an attribute causes the change of an entire image). By contrast, this work introduces low-rank subspaces that enable more precise control of GAN generation. Concretely, given an arbitrary image and a region of interest (e.g., eyes of face images), we manage to relate the latent space to the image region with the Jacobian matrix and then use low-rank factorization to discover steerable latent subspaces. There are three distinguishable strengths of our approach that can be aptly called LowRankGAN. First, compared to analytic algorithms in prior work, our low-rank factorization of Jacobians is able to find the low-dimensional representation of attribute manifold, making image editing more precise and controllable. Second, low-rank factorization naturally yields a null space of attributes such that moving the latent code within it only affects the outer region of interest. Therefore, local image editing can be simply achieved by projecting an attribute vector into the null space without relying on a spatial mask as existing methods do. Third, our method can robustly work with a local region from one image for analysis yet well generalize to other images, making it much easy to use in practice. Extensive experiments on state-of-the-art GAN models (including StyleGAN2 and BigGAN) trained on various datasets demonstrate the effectiveness of our LowRankGAN.
Author Information
Jiapeng Zhu (Hong Kong University of Science and Technology)
Ruili Feng (University of Science and Technology of China)
Yujun Shen (ByteDance Inc.)
Deli Zhao (Xiaomi AI Lab)
Zheng-Jun Zha (University of Science and Technology of China)
Jingren Zhou (Alibaba Group)
Qifeng Chen (HKUST)
More from the Same Authors
-
2022 Poster: Exploring Figure-Ground Assignment Mechanism in Perceptual Organization »
Wei Zhai · Yang Cao · Jing Zhang · Zheng-Jun Zha -
2022 Poster: Stochastic Window Transformer for Image Restoration »
Jie Xiao · Xueyang Fu · Feng Wu · Zheng-Jun Zha -
2022 Poster: Rank Diminishing in Deep Neural Networks »
Ruili Feng · Kecheng Zheng · Yukun Huang · Deli Zhao · Michael Jordan · Zheng-Jun Zha -
2023 Poster: FaceComposer: A Unified Framework for Versatile Facial Content Creation »
Jiayu Wang · Kang Zhao · Yifeng Ma · Shiwei Zhang · Yingya Zhang · Yujun Shen · Deli Zhao · Jingren Zhou -
2023 Poster: Learning Modulated Transformation in GANs »
Ceyuan Yang · Qihang Zhang · Yinghao Xu · Jiapeng Zhu · Yujun Shen · Bo Dai -
2023 Poster: VideoComposer: Compositional Video Synthesis with Motion Controllability »
Xiang Wang · Hangjie Yuan · Shiwei Zhang · Dayou Chen · Jiuniu Wang · Yingya Zhang · Yujun Shen · Deli Zhao · Jingren Zhou -
2023 Poster: Customizable Image Synthesis with Multiple Subjects »
Zhiheng Liu · Yifei Zhang · Yujun Shen · Kecheng Zheng · Kai Zhu · Ruili Feng · Yu Liu · Deli Zhao · Jingren Zhou · Yang Cao -
2023 Poster: Res-Tuning: A Flexible and Efficient Tuning Paradigm via Unbinding Tuner from Backbone »
Zeyinzi Jiang · Chaojie Mao · Ziyuan Huang · Ao Ma · Yiliang Lv · Yujun Shen · Deli Zhao · Jingren Zhou -
2023 Poster: DreamWaltz: Make a Scene with Complex 3D Animatable Avatars »
Yukun Huang · Jianan Wang · Ailing Zeng · He CAO · Xianbiao Qi · Yukai Shi · Zheng-Jun Zha · Lei Zhang -
2023 Poster: MomentDiff: Generative Video Moment Retrieval from Random to Real »
Pandeng Li · Chen-Wei Xie · Hongtao Xie · Liming Zhao · Lei Zhang · Yun Zheng · Deli Zhao · Yongdong Zhang -
2023 Poster: DIFFER:Decomposing Individual Reward for Fair Experience Replay in Multi-Agent Reinforcement Learning »
Xunhan Hu · Jian Zhao · Wengang Zhou · Ruili Feng · Houqiang Li -
2022 Spotlight: Lightning Talks 5B-3 »
Yanze Wu · Jie Xiao · Nianzu Yang · Jieyi Bi · Jian Yao · Yiting Chen · Qizhou Wang · Yangru Huang · Yongqiang Chen · Peixi Peng · Yuxin Hong · Xintao Wang · Feng Liu · Yining Ma · Qibing Ren · Xueyang Fu · Yonggang Zhang · Kaipeng Zeng · Jiahai Wang · GEN LI · Yonggang Zhang · Qitian Wu · Yifan Zhao · Chiyu Wang · Junchi Yan · Feng Wu · Yatao Bian · Xiaosong Jia · Ying Shan · Zhiguang Cao · Zheng-Jun Zha · Guangyao Chen · Tianjun Xiao · Han Yang · Jing Zhang · Jinbiao Chen · MA Kaili · Yonghong Tian · Junchi Yan · Chen Gong · Tong He · Binghui Xie · Yuan Sun · Francesco Locatello · Tongliang Liu · Yeow Meng Chee · David P Wipf · Tongliang Liu · Bo Han · Bo Han · Yanwei Fu · James Cheng · Zheng Zhang -
2022 Spotlight: Improving 3D-aware Image Synthesis with A Geometry-aware Discriminator »
Zifan Shi · Yinghao Xu · Yujun Shen · Deli Zhao · Qifeng Chen · Dit-Yan Yeung -
2022 Spotlight: Stochastic Window Transformer for Image Restoration »
Jie Xiao · Xueyang Fu · Feng Wu · Zheng-Jun Zha -
2022 Spotlight: Lightning Talks 5B-1 »
Devansh Arpit · Xiaojun Xu · Zifan Shi · Ivan Skorokhodov · Shayan Shekarforoush · Zhan Tong · Yiqun Wang · Shichong Peng · Linyi Li · Ivan Skorokhodov · Huan Wang · Yibing Song · David Lindell · Yinghao Xu · Seyed Alireza Moazenipourasil · Sergey Tulyakov · Peter Wonka · Yiqun Wang · Ke Li · David Fleet · Yujun Shen · Yingbo Zhou · Bo Li · Jue Wang · Peter Wonka · Marcus Brubaker · Caiming Xiong · Limin Wang · Deli Zhao · Qifeng Chen · Dit-Yan Yeung -
2022 Poster: Improving 3D-aware Image Synthesis with A Geometry-aware Discriminator »
Zifan Shi · Yinghao Xu · Yujun Shen · Deli Zhao · Qifeng Chen · Dit-Yan Yeung -
2022 Poster: Improving GANs with A Dynamic Discriminator »
Ceyuan Yang · Yujun Shen · Yinghao Xu · Deli Zhao · Bo Dai · Bolei Zhou -
2021 Poster: Data-Efficient Instance Generation from Instance Discrimination »
Ceyuan Yang · Yujun Shen · Yinghao Xu · Bolei Zhou -
2021 Poster: UFC-BERT: Unifying Multi-Modal Controls for Conditional Image Synthesis »
Zhu Zhang · Jianxin Ma · Chang Zhou · Rui Men · Zhikang Li · Ming Ding · Jie Tang · Jingren Zhou · Hongxia Yang -
2020 Poster: Hierarchical Granularity Transfer Learning »
Shaobo Min · Hongtao Xie · Hantao Yao · Xuran Deng · Zheng-Jun Zha · Yongdong Zhang -
2020 Poster: Blind Video Temporal Consistency via Deep Video Prior »
Chenyang Lei · Yazhou Xing · Qifeng Chen -
2020 Poster: Learning to Mutate with Hypergradient Guided Population »
Zhiqiang Tao · Yaliang Li · Bolin Ding · Ce Zhang · Jingren Zhou · Yun Fu -
2020 Poster: Learning Semantic-aware Normalization for Generative Adversarial Networks »
Heliang Zheng · Jianlong Fu · Yanhong Zeng · Jiebo Luo · Zheng-Jun Zha -
2020 Spotlight: Learning Semantic-aware Normalization for Generative Adversarial Networks »
Heliang Zheng · Jianlong Fu · Yanhong Zeng · Jiebo Luo · Zheng-Jun Zha -
2019 Poster: Learning Deep Bilinear Transformation for Fine-grained Image Representation »
Heliang Zheng · Jianlong Fu · Zheng-Jun Zha · Jiebo Luo -
2019 Poster: Abstract Reasoning with Distracting Features »
Kecheng Zheng · Zheng-Jun Zha · Wei Wei -
2018 Poster: Combinatorial Optimization with Graph Convolutional Networks and Guided Tree Search »
Zhuwen Li · Qifeng Chen · Vladlen Koltun -
2018 Poster: DeepExposure: Learning to Expose Photos with Asynchronously Reinforced Adversarial Learning »
Runsheng Yu · Wenyu Liu · Yasen Zhang · Zhi Qu · Deli Zhao · Bo Zhang -
2014 Poster: Zeta Hull Pursuits: Learning Nonconvex Data Hulls »
Yuanjun Xiong · Wei Liu · Deli Zhao · Xiaoou Tang -
2008 Poster: Cyclizing Clusters via Zeta Function of a Graph »
Deli Zhao · Xiaoou Tang