Convolution network pruning based on the evaluation of the importance of characteristic attributions

Although deep learning models have recently achieved remarkable performance in many tasks, they require massive memory footprint and computing power to achieve efficient inference. The researchers propose a number of compression methods to compress the capacity and computation of the model so that deep learning can be deployed to resource-constrained mobile terminals. Based on the pruning framework, two pruning methods are proposed from the perspective of filter importance evaluation. (1) As every filter can learn unique features, we propose an attribution mechanism to evaluate the correlation between the features learned by a filter and the causal features. We prune the filter with low correlation so as to compress the model and retain the attribution characteristics of the original model; the process is called attribution pruning. (2) The second pruning method uses positive correlation features in a channel and gradient to evaluate the importance of the filter, which is based on an iterative optimization pruning framework. This method, which is called Taylor-guided pruning, can improve the accuracy of pruning redundant filters. We implement two pruning methods in VGGNet and ResNet. Extensive experiments demonstrate that attribution pruning can greatly retain the attribution characteristics of the original model. Moreover, the two pruning methods can achieve better compression than current mainstream pruning methods.