Outlier-Aware Training for Low-Bit Quantization of Structural
  Re-Parameterized Networks
Muqun Niu Yuan Ren Boyu Li Chenchen Ding
Lightweight design of Convolutional Neural Networks (CNNs) requires co-design
efforts in the model architectures and compression techniques. As a novel
design paradigm that separates training and inference, a structural
re-parameterized (SR) network such as the representative RepVGG revitalizes the
simple VGG-like network with a high accuracy comparable to advanced and often
more complicated networks. However, the merging process in SR networks
introduces outliers into weights, making their distribution distinct from
conventional networks and thus heightening difficulties in quantization. To
address this, we propose an operator-level improvement for training called
Outlier Aware Batch Normalization (OABN). Additionally, to meet the demands of
limited bitwidths while upkeeping the inference accuracy, we develop a
clustering-based non-uniform quantization framework for Quantization-Aware
Training (QAT) named ClusterQAT. Integrating OABN with ClusterQAT, the
quantized performance of RepVGG is largely enhanced, particularly when the
bitwidth falls below 8.