Timezone: »

 
Poster
Accelerating Sparse Convolution with Column Vector-Wise Sparsity
Yijun Tan · Kai Han · Kang Zhao · Xianzhi Yu · Zidong Du · Yunji Chen · Yunhe Wang · Jun Yao

Wed Nov 30 02:00 PM -- 04:00 PM (PST) @ Hall J #620
in Poster Session 4 »
Weight sparsity is a promising approach to reducing the model size and computation cost of convolutional neural networks (CNNs). Nevertheless, non-zero weights often distribute randomly in sparse CNN models, introducing enormous difficulty in obtaining actual speedup on common hardware (e.g., GPU) over their dense counterparts. Existing acceleration solutions either require hardware modifications for irregular memory access support or rely on a partially structured sparsity pattern. Neither of these methods is capable of achieving fruitful speedup on convolution layers.In this work, we propose an algorithm-software co-designed sparse convolution based on a novel out-vector-wise (OVW) sparse pattern. Building on the insight that vertical vector integrity can preserve continuous memory access in IM2COL, the OVW pattern treats a $V\times1$ vector as an entirety. To reduce the error caused by sparsity, we propose an equivalent transformation process, i.e., clustering-based channel permutation, to gather similar rows together. Experimental evaluations demonstrate that our method achieves a $1.7\times$ and $3.2\times$ speedup over the SOTA solution and the dense convolution of ResNet50 on NVIDIA V100 at 75\% sparsity, respectively, with only negligible accuracy loss. Moreover, compared to the SOTA solution that achieves speedups only on data with 60\% sparsity or more, our method begins to obtain speedups on data with only 10\% sparsity.
[ Paper]  [ Poster]  [ OpenReview] 

Author Information

Yijun Tan (Chinese Academy of Sciences)
Kai Han (Huawei Noah's Ark Lab)
Kang Zhao (Huawei Technologies Ltd.)
Xianzhi Yu (University of the Chinese Academy of Sciences)
Zidong Du (Institute of Computing Technology, Chinese Academy of Sciences)
Yunji Chen (Institute of Computing Technology, Chinese Academy of Sciences)
Yunhe Wang (Huawei Noah's Ark Lab)
Jun Yao (Huawei Tech R&D UK)

More from the Same Authors