Timezone: »

Q-DM: An Efficient Low-bit Quantized Diffusion Model
Yanjing Li · Sheng Xu · Xianbin Cao · Xiao Sun · Baochang Zhang

Thu Dec 14 08:45 AM -- 10:45 AM (PST) @ Great Hall & Hall B1+B2 #122

Denoising diffusion generative models are capable of generating high-quality data, but suffers from the computation-costly generation process, due to a iterative noise estimation using full-precision networks. As an intuitive solution, quantization can significantly reduce the computational and memory consumption by low-bit parameters and operations. However, low-bit noise estimation networks in diffusion models (DMs) remain unexplored yet and perform much worse than the full-precision counterparts as observed in our experimental studies. In this paper, we first identify that the bottlenecks of low-bit quantized DMs come from a large distribution oscillation on activations and accumulated quantization error caused by the multi-step denoising process. To address these issues, we first develop a Timestep-aware Quantization (TaQ) method and a Noise-estimating Mimicking (NeM) scheme for low-bit quantized DMs (Q-DM) to effectively eliminate such oscillation and accumulated error respectively, leading to well-performed low-bit DMs. In this way, we propose an efficient Q-DM to calculate low-bit DMs by considering both training and inference process in the same framework. We evaluate our methods on popular DDPM and DDIM models. Extensive experimental results show that our method achieves a much better performance than the prior arts. For example, the 4-bit Q-DM theoretically accelerates the 1000-step DDPM by 7.8x and achieves a FID score of 5.17, on the unconditional CIFAR-10 dataset.

Author Information

Yanjing Li (Beihang University)
Sheng Xu (Beihang University)
Xianbin Cao (Beihang University)
Xiao Sun (Microsoft Research)

Xiao is a scientist with Shanghai AI Laboratory, co-leading a research group on AI for Sports and Human-Centered Computing. Before that, he worked as a Senior Researcher at the Visual Computing Group, Microsoft Research Asia (MSRA), from Feb. 2016 to Jul. 2022. His research interests include computer vision, machine learning, and computer graphics. Xiao received his B.S. and M.S. degrees from the South China University of Technology in 2011 and 2014, respectively. After that, he joined the Multimedia Lab, Department of Information Engineering, the Chinese University of Hong Kong as a research associate from 2014 to 2015. Job opening: I'm actively looking for researchers, engineers, and interns working in Beijing or Shanghai. If you are interested, please send me an email (sunxiao@pjlab.org.cn).

Baochang Zhang (Beihang University)

More from the Same Authors