Frequency-Domain Dynamic Pruning For Convolutional Neural Networks

Authors:
Zhenhua Liu Peking University
Jizheng Xu Bytedance Inc.
Xiulian Peng Microsoft Research
Ruiqin Xiong Peking University

Introduction:

Deep convolutional neural networks have demonstrated their powerfulness in a variety of applications.Considering that there are spatial redundancy within most filters in a CNN, the authors propose a frequency-domain dynamic pruning scheme to exploit the spatial correlations.

Abstract:

Deep convolutional neural networks have demonstrated their powerfulness in a variety of applications. However, the storage and computational requirements have largely restricted their further extensions on mobile devices. Recently, pruning of unimportant parameters has been used for both network compression and acceleration. Considering that there are spatial redundancy within most filters in a CNN, we propose a frequency-domain dynamic pruning scheme to exploit the spatial correlations. The frequency-domain coefficients are pruned dynamically in each iteration and different frequency bands are pruned discriminatively, given their different importance on accuracy. Experimental results demonstrate that the proposed scheme can outperform previous spatial-domain counterparts by a large margin. Specifically, it can achieve a compression ratio of 8.4x and a theoretical inference speed-up of 9.2x for ResNet-110, while the accuracy is even better than the reference model on CIFAR-110.

You may want to know: