Exploring Quantization and Mapping Synergy in Hardware-Aware Deep Neural
  Network Accelerators
Jan Klhufek Miroslav Safar Vojtech Mrazek Zdenek Vasicek Lukas Sekanina
Energy efficiency and memory footprint of a convolutional neural network
(CNN) implemented on a CNN inference accelerator depend on many factors,
including a weight quantization strategy (i.e., data types and bit-widths) and
mapping (i.e., placement and scheduling of DNN elementary operations on
hardware units of the accelerator). We show that enabling rich mixed
quantization schemes during the implementation can open a previously hidden
space of mappings that utilize the hardware resources more effectively. CNNs
utilizing quantized weights and activations and suitable mappings can
significantly improve trade-offs among the accuracy, energy, and memory
requirements compared to less carefully optimized CNN implementations. To find,
analyze, and exploit these mappings, we: (i) extend a general-purpose
state-of-the-art mapping tool (Timeloop) to support mixed quantization, which
is not currently available; (ii) propose an efficient multi-objective
optimization algorithm to find the most suitable bit-widths and mapping for
each DNN layer executed on the accelerator; and (iii) conduct a detailed
experimental evaluation to validate the proposed method. On two CNNs
(MobileNetV1 and MobileNetV2) and two accelerators (Eyeriss and Simba) we show
that for a given quality metric (such as the accuracy on ImageNet), energy
savings are up to 37% without any accuracy drop.