Chain of Compression: A Systematic Approach to Combinationally Compress
  Convolutional Neural Networks
Yingtao Shen Minqing Sun Jie Zhao An Zou
Convolutional neural networks (CNNs) have achieved significant popularity,
but their computational and memory intensity poses challenges for
resource-constrained computing systems, particularly with the prerequisite of
real-time performance. To release this burden, model compression has become an
important research focus. Many approaches like quantization, pruning, early
exit, and knowledge distillation have demonstrated the effect of reducing
redundancy in neural networks. Upon closer examination, it becomes apparent
that each approach capitalizes on its unique features to compress the neural
network, and they can also exhibit complementary behavior when combined. To
explore the interactions and reap the benefits from the complementary features,
we propose the Chain of Compression, which works on the combinational sequence
to apply these common techniques to compress the neural network. Validated on
the image-based regression and classification networks across different data
sets, our proposed Chain of Compression can significantly compress the
computation cost by 100-1000 times with ignorable accuracy loss compared with
the baseline model.