Timezone: »

 
Poster
Focused Quantization for Sparse CNNs
Yiren Zhao · Xitong Gao · Daniel Bates · Robert Mullins · Cheng-Zhong Xu

Wed Dec 11 10:45 AM -- 12:45 PM (PST) @ East Exhibition Hall B + C #101
in Deep Learning -- Efficient Inference Methods »

Deep convolutional neural networks (CNNs) are powerful tools for a wide range of vision tasks, but the enormous amount of memory and compute resources required by CNNs poses a challenge in deploying them on constrained devices. Existing compression techniques, while excelling at reducing model sizes, struggle to be computationally friendly. In this paper, we attend to the statistical properties of sparse CNNs and present focused quantization, a novel quantization strategy based on power-of-two values, which exploits the weight distributions after fine-grained pruning. The proposed method dynamically discovers the most effective numerical representation for weights in layers with varying sparsities, significantly reducing model sizes. Multiplications in quantized CNNs are replaced with much cheaper bit-shift operations for efficient inference. Coupled with lossless encoding, we build a compression pipeline that provides CNNs with high compression ratios (CR), low computation cost and minimal loss in accuracies. In ResNet-50, we achieved a 18.08x CR with only 0.24% loss in top-5 accuracy, outperforming existing compression methods. We fully compress a ResNet-18 and found that it is not only higher in CR and top-5 accuracy, but also more hardware efficient as it requires fewer logic gates to implement when compared to other state-of-the-art quantization methods assuming the same throughput.

Keywords: Efficient Inference Methods · CNN Architectures · Deep Learning · Applications -> Computer Vision; Deep Learning
[ Paper]  [ Poster] 

Author Information

Yiren Zhao (University of Cambridge)
Xitong Gao (Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences)
Daniel Bates (University of Cambridge)
Robert Mullins (University of Cambridge)
Cheng-Zhong Xu (University of Macau)

More from the Same Authors