Pruning Neural Networks with Velocity-Constrained Optimization

Pruning has gained prominence as a way to compress over-parameterized neural networks. Whilepruning can be understood as solving a sparsity-constrained optimization problem, pruning by di-rectly solving this problem has been relatively underexplored. In this paper, we propose a method toprune neural networks using the MJ algorithm, which interprets constrained optimization using theframework of velocity-constrained optimization. The experimental results show that our methodcan prune VGG19 and ResNet32 networks by more than 90% while preserving the high accuracyof the dense network.