Reducing the Side-Effects of Oscillations in Training of Quantized YOLO
  Networks
Kartik Gupta Akshay Asthana
Quantized networks use less computational and memory resources and are
suitable for deployment on edge devices. While quantization-aware training QAT
is the well-studied approach to quantize the networks at low precision, most
research focuses on over-parameterized networks for classification with limited
studies on popular and edge device friendly single-shot object detection and
semantic segmentation methods like YOLO. Moreover, majority of QAT methods rely
on Straight-through Estimator (STE) approximation which suffers from an
oscillation phenomenon resulting in sub-optimal network quantization. In this
paper, we show that it is difficult to achieve extremely low precision (4-bit
and lower) for efficient YOLO models even with SOTA QAT methods due to
oscillation issue and existing methods to overcome this problem are not
effective on these models. To mitigate the effect of oscillation, we first
propose Exponentially Moving Average (EMA) based update to the QAT model.
Further, we propose a simple QAT correction method, namely QC, that takes only
a single epoch of training after standard QAT procedure to correct the error
induced by oscillating weights and activations resulting in a more accurate
quantized model. With extensive evaluation on COCO dataset using various YOLO5
and YOLO7 variants, we show that our correction method improves quantized YOLO
networks consistently on both object detection and segmentation tasks at
low-precision (4-bit and 3-bit).