EPTQ: Enhanced Post-Training Quantization via Label-Free Hessian
Ofir Gordon Hai Victor Habi Arnon Netzer
Quantization of deep neural networks (DNN) has become a key element in the
efforts of embedding such networks on end-user devices. However, current
quantization methods usually suffer from costly accuracy degradation. In this
paper, we propose a new method for Enhanced Post Training Quantization named
EPTQ. The method is based on knowledge distillation with an adaptive weighting
of layers. In addition, we introduce a new label-free technique for
approximating the Hessian trace of the task loss, named Label-Free Hessian.
This technique removes the requirement of a labeled dataset for computing the
Hessian. The adaptive knowledge distillation uses the Label-Free Hessian
technique to give greater attention to the sensitive parts of the model while
performing the optimization. Empirically, by employing EPTQ we achieve
state-of-the-art results on a wide variety of models, tasks, and datasets,
including ImageNet classification, COCO object detection, and Pascal-VOC for
semantic segmentation. We demonstrate the performance and compatibility of EPTQ
on an extended set of architectures, including CNNs, Transformers, hybrid, and
MLP-only models.