Timezone: »
Although two-stage Vector Quantized (VQ) generative models allow for synthesizing high-fidelity and high-resolution images, their quantization operator encodes similar patches within an image into the same index, resulting in a repeated artifact for similar adjacent regions using existing decoder architectures. To address this issue, we propose to incorporate the spatially conditional normalization to modulate the quantized vectors so as to insert spatially variant information to the embedded index maps, encouraging the decoder to generate more photorealistic images. Moreover, we use multichannel quantization to increase the recombination capability of the discrete codes without increasing the cost of model and codebook. Additionally, to generate discrete tokens at the second stage, we adopt a Masked Generative Image Transformer (MaskGIT) to learn an underlying prior distribution in the compressed latent space, which is much faster than the conventional autoregressive model. Experiments on two benchmark datasets demonstrate that our proposed modulated VQGAN is able to greatly improve the reconstructed image quality as well as provide high-fidelity image generation.
Author Information
Chuanxia Zheng (Monash University)
I am a postdoctoral researcher in the VGG at the University of Oxford, working with Prof. Andrea Vedaldi. Before that, I spent one year at Monash University, where I worked as a Research Fellow with Prof. Jianfei Cai and Prof. Dinh Phung. I received my PhD degree from the SCSE at Nanyang Technological University, supervised by Prof. Tat-Jen Cham and Prof. Jianfei Cai. My thesis Synthesizing Photorealistic Images was awarded the NTU Outstanding PhD Thesis Award 2022.
Tung-Long Vuong (VNU - University of Engineering and Technology)
Jianfei Cai (Monash University)
Dinh Phung (Monash University)
More from the Same Authors
-
2023 Poster: Cocktail: Mixing Multi-Modality Control for Text-Conditional Image Generation »
Minghui Hu · Jianbin Zheng · Daqing Liu · Chuanxia Zheng · Chaoyue Wang · Dacheng Tao · Tat-Jen Cham -
2023 Poster: Flat Seeking Bayesian Neural Networks »
Van-Anh Nguyen · Tung-Long Vuong · Hoang Phan · Thanh-Toan Do · Dinh Phung · Trung Le -
2023 Poster: Optimal Transport Model Distributional Robustness »
Van-Anh Nguyen · Trung Le · Anh Bui · Thanh-Toan Do · Dinh Phung -
2023 Poster: Model and Feature Diversity for Bayesian Neural Networks in Mutual Learning »
Van Cuong Pham · Cuong C Nguyen · Trung Le · Dinh Phung · Gustavo Carneiro · Thanh-Toan Do -
2022 Spotlight: Lightning Talks 6B-4 »
Junjie Chen · Chuanxia Zheng · JINLONG LI · Yu Shi · Shichao Kan · Yu Wang · Fermín Travi · Ninh Pham · Lei Chai · Guobing Gan · Tung-Long Vuong · Gonzalo Ruarte · Tao Liu · Li Niu · Jingjing Zou · Zequn Jie · Peng Zhang · Ming LI · Yixiong Liang · Guolin Ke · Jianfei Cai · Gaston Bujia · Sunzhu Li · Siyuan Zhou · Jingyang Lin · Xu Wang · Min Li · Zhuoming Chen · Qing Ling · Xiaolin Wei · Xiuqing Lu · Shuxin Zheng · Dinh Phung · Yigang Cen · Jianlou Si · Juan Esteban Kamienkowski · Jianxin Wang · Chen Qian · Lin Ma · Benyou Wang · Yingwei Pan · Tie-Yan Liu · Liqing Zhang · Zhihai He · Ting Yao · Tao Mei -
2022 Spotlight: Lightning Talks 6B-3 »
Lingfeng Yang · Yao Lai · Zizheng Pan · Zhenyu Wang · Weicong Liang · Chuanyang Zheng · Jian-Wei Zhang · Peng Jin · Jing Liu · Xiuying Wei · Yao Mu · Xiang Li · YUHUI YUAN · Zizheng Pan · Yifan Sun · Yunchen Zhang · Jianfei Cai · Hao Luo · zheyang li · Jinfa Huang · Haoyu He · Yi Yang · Ping Luo · Fenglin Liu · Henghui Ding · Borui Zhao · Xiangguo Zhang · Kai Zhang · Pichao WANG · Bohan Zhuang · Wei Chen · Ruihao Gong · Zhi Yang · Xian Wu · Feng Ding · Jianfei Cai · Xiao Luo · Renjie Song · Weihong Lin · Jian Yang · Wenming Tan · Bohan Zhuang · Shanghang Zhang · Shen Ge · Fan Wang · Qi Zhang · Guoli Song · Jun Xiao · Hao Li · Ding Jia · David Clifton · Ye Ren · Fengwei Yu · Zheng Zhang · Jie Chen · Shiliang Pu · Xianglong Liu · Chao Zhang · Han Hu -
2022 Spotlight: EcoFormer: Energy-Saving Attention with Linear Complexity »
Jing Liu · Zizheng Pan · Haoyu He · Jianfei Cai · Bohan Zhuang -
2022 Spotlight: Fast Vision Transformers with HiLo Attention »
Zizheng Pan · Jianfei Cai · Bohan Zhuang -
2022 Spotlight: MoVQ: Modulating Quantized Vectors for High-Fidelity Image Generation »
Chuanxia Zheng · Tung-Long Vuong · Jianfei Cai · Dinh Phung -
2022 Poster: Fast Vision Transformers with HiLo Attention »
Zizheng Pan · Jianfei Cai · Bohan Zhuang -
2022 Poster: Stochastic Multiple Target Sampling Gradient Descent »
Hoang Phan · Ngoc Tran · Trung Le · Toan Tran · Nhat Ho · Dinh Phung -
2022 Poster: EcoFormer: Energy-Saving Attention with Linear Complexity »
Jing Liu · Zizheng Pan · Haoyu He · Jianfei Cai · Bohan Zhuang -
2021 Poster: Exploiting Domain-Specific Features to Enhance Domain Generalization »
Manh-Ha Bui · Toan Tran · Anh Tran · Dinh Phung -
2021 Poster: On Learning Domain-Invariant Representations for Transfer Learning with Multiple Sources »
Trung Phung · Trung Le · Tung-Long Vuong · Toan Tran · Anh Tran · Hung Bui · Dinh Phung -
2020 : QuatRE: Relation-Aware Quaternions for Knowledge Graph Embeddings »
Dai Quoc Nguyen · Dinh Phung -
2020 : Quaternion Graph Neural Networks »
Dai Quoc Nguyen · Tu Dinh Nguyen · Dinh Phung -
2020 Poster: OTLDA: A Geometry-aware Optimal Transport Approach for Topic Modeling »
Viet Huynh · He Zhao · Dinh Phung -
2019 : Poster session »
Sebastian Farquhar · Erik Daxberger · Andreas Look · Matt Benatan · Ruiyi Zhang · Marton Havasi · Fredrik Gustafsson · James A Brofos · Nabeel Seedat · Micha Livne · Ivan Ustyuzhaninov · Adam Cobb · Felix D McGregor · Patrick McClure · Tim R. Davidson · Gaurush Hiranandani · Sanjeev Arora · Masha Itkina · Didrik Nielsen · William Harvey · Matias Valdenegro-Toro · Stefano Peluchetti · Riccardo Moriconi · Tianyu Cui · Vaclav Smidl · Taylan Cemgil · Jack Fitzsimons · He Zhao · · mariana vargas vieyra · Apratim Bhattacharyya · Rahul Sharma · Geoffroy Dubourg-Felonneau · Jonathan Warrell · Slava Voloshynovskiy · Mihaela Rosca · Jiaming Song · Andrew Ross · Homa Fashandi · Ruiqi Gao · Hooshmand Shokri Razaghi · Joshua Chang · Tim Xiao · Vanessa Boehm · Giorgio Giannone · Ranganath Krishnan · Joe Davison · Arsenii Ashukha · Jeremiah Liu · Sicong (Sheldon) Huang · Evgenii Nikishin · Sunho Park · Nilesh Ahuja · Mahesh Subedar · · Artyom Gadetsky · Jhosimar Arias Figueroa · Tim G. J. Rudner · Waseem Aslam · Adrián Csiszárik · John Moberg · Ali Hebbal · Kathrin Grosse · Pekka Marttinen · Bang An · Hlynur Jónsson · Samuel Kessler · Abhishek Kumar · Mikhail Figurnov · Omesh Tickoo · Steindor Saemundsson · Ari Heljakka · Dániel Varga · Niklas Heim · Simone Rossi · Max Laves · Waseem Gharbieh · Nicholas Roberts · Luis Armando Pérez Rey · Matthew Willetts · Prithvijit Chakrabarty · Sumedh Ghaisas · Carl Shneider · Wray Buntine · Kamil Adamczewski · Xavier Gitiaux · Suwen Lin · Hao Fu · Gunnar Rätsch · Aidan Gomez · Erik Bodin · Dinh Phung · Lennart Svensson · Juliano Tusi Amaral Laganá Pinto · Milad Alizadeh · Jianzhun Du · Kevin Murphy · Beatrix Benkő · Shashaank Vattikuti · Jonathan Gordon · Christopher Kanan · Sontje Ihler · Darin Graham · Michael Teng · Louis Kirsch · Tomas Pevny · Taras Holotyak