Adaptive quantization with mixed-precision based on low-cost proxy
Junzhe Chen Qiao Yang Senmao Tian Shunli Zhang
It is critical to deploy complicated neural network models on hardware with
limited resources. This paper proposes a novel model quantization method, named
the Low-Cost Proxy-Based Adaptive Mixed-Precision Model Quantization (LCPAQ),
which contains three key modules. The hardware-aware module is designed by
considering the hardware limitations, while an adaptive mixed-precision
quantization module is developed to evaluate the quantization sensitivity by
using the Hessian matrix and Pareto frontier techniques. Integer linear
programming is used to fine-tune the quantization across different layers. Then
the low-cost proxy neural architecture search module efficiently explores the
ideal quantization hyperparameters. Experiments on the ImageNet demonstrate
that the proposed LCPAQ achieves comparable or superior quantization accuracy
to existing mixed-precision models. Notably, LCPAQ achieves 1/200 of the search
time compared with existing methods, which provides a shortcut in practical
quantization use for resource-limited devices.