Structured Network Pruning by Measuring Filter-wise Interactions
Wenting Tang Xingxing Wei Bo Li
Structured network pruning is a practical approach to reduce computation cost
directly while retaining the CNNs' generalization performance in real
applications. However, identifying redundant filters is a core problem in
structured network pruning, and current redundancy criteria only focus on
individual filters' attributes. When pruning sparsity increases, these
redundancy criteria are not effective or efficient enough. Since the
filter-wise interaction also contributes to the CNN's prediction accuracy, we
integrate the filter-wise interaction into the redundancy criterion. In our
criterion, we introduce the filter importance and filter utilization strength
to reflect the decision ability of individual and multiple filters. Utilizing
this new redundancy criterion, we propose a structured network pruning approach
SNPFI (Structured Network Pruning by measuring Filter-wise Interaction). During
the pruning, the SNPFI can automatically assign the proper sparsity based on
the filter utilization strength and eliminate the useless filters by filter
importance. After the pruning, the SNPFI can recover pruned model's performance
effectively without iterative training by minimizing the interaction
difference. We empirically demonstrate the effectiveness of the SNPFI with
several commonly used CNN models, including AlexNet, MobileNetv1, and
ResNet-50, on various image classification datasets, including MNIST, CIFAR-10,
and ImageNet. For all experimental CNN models, nearly 60% of computation is
reduced in a network compression while the classification accuracy remains.