A Structured Pruning Algorithm for Model-based Deep Learning
Chicago Park Weijie Gan Zihao Zou Yuyang Hu Zhixin Sun Ulugbek S. Kamilov
There is a growing interest in model-based deep learning (MBDL) for solving
imaging inverse problems. MBDL networks can be seen as iterative algorithms
that estimate the desired image using a physical measurement model and a
learned image prior specified using a convolutional neural net (CNNs). The
iterative nature of MBDL networks increases the test-time computational
complexity, which limits their applicability in certain large-scale
applications. We address this issue by presenting structured pruning algorithm
for model-based deep learning (SPADE) as the first structured pruning algorithm
for MBDL networks. SPADE reduces the computational complexity of CNNs used
within MBDL networks by pruning its non-essential weights. We propose three
distinct strategies to fine-tune the pruned MBDL networks to minimize the
performance loss. Each fine-tuning strategy has a unique benefit that depends
on the presence of a pre-trained model and a high-quality ground truth. We
validate SPADE on two distinct inverse problems, namely compressed sensing MRI
and image super-resolution. Our results highlight that MBDL models pruned by
SPADE can achieve substantial speed up in testing time while maintaining
competitive performance.