Post-Training Quantization for Re-parameterization via Coarse & Fine
  Weight Splitting
Dawei Yang Ning He Xing Hu Zhihang Yuan Jiangyong Yu Chen Xu Zhe Jiang
Although neural networks have made remarkable advancements in various
applications, they require substantial computational and memory resources.
Network quantization is a powerful technique to compress neural networks,
allowing for more efficient and scalable AI deployments. Recently,
Re-parameterization has emerged as a promising technique to enhance model
performance while simultaneously alleviating the computational burden in
various computer vision tasks. However, the accuracy drops significantly when
applying quantization on the re-parameterized networks. We identify that the
primary challenge arises from the large variation in weight distribution across
the original branches. To address this issue, we propose a coarse & fine weight
splitting (CFWS) method to reduce quantization error of weight, and develop an
improved KL metric to determine optimal quantization scales for activation. To
the best of our knowledge, our approach is the first work that enables
post-training quantization applicable on re-parameterized networks. For
example, the quantized RepVGG-A1 model exhibits a mere 0.3% accuracy loss. The
code is in https://github.com/NeonHo/Coarse-Fine-Weight-Split.git