DNN Memory Footprint Reduction via Post-Training Intra-Layer
  Multi-Precision Quantization
Behnam Ghavami Amin Kamjoo Lesley Shannon Steve Wilton
The imperative to deploy Deep Neural Network (DNN) models on
resource-constrained edge devices, spurred by privacy concerns, has become
increasingly apparent. To facilitate the transition from cloud to edge
computing, this paper introduces a technique that effectively reduces the
memory footprint of DNNs, accommodating the limitations of resource-constrained
edge devices while preserving model accuracy. Our proposed technique, named
Post-Training Intra-Layer Multi-Precision Quantization (PTILMPQ), employs a
post-training quantization approach, eliminating the need for extensive
training data. By estimating the importance of layers and channels within the
network, the proposed method enables precise bit allocation throughout the
quantization process. Experimental results demonstrate that PTILMPQ offers a
promising solution for deploying DNNs on edge devices with restricted memory
resources. For instance, in the case of ResNet50, it achieves an accuracy of
74.57\% with a memory footprint of 9.5 MB, representing a 25.49\% reduction
compared to previous similar methods, with only a minor 1.08\% decrease in
accuracy.