RepQuant: Towards Accurate Post-Training Quantization of Large
  Transformer Models via Scale Reparameterization
Zhikai Li Xuewen Liu Jing Zhang Qingyi Gu
Large transformer models have demonstrated remarkable success. Post-training
quantization (PTQ), which requires only a small dataset for calibration and
avoids end-to-end retraining, is a promising solution for compressing these
large models. Regrettably, existing PTQ methods typically exhibit non-trivial
performance loss. We find that the performance bottleneck stems from
over-consideration of hardware compatibility in the quantization process,
compelling them to reluctantly employ simple quantizers, albeit at the expense
of accuracy. With the above insights, we propose RepQuant, a novel PTQ
framework with quantization-inference decoupling paradigm to address the above
issues. RepQuant employs complex quantizers in the quantization process and
simplified quantizers in the inference process, and performs mathematically
equivalent transformations between the two through quantization scale
reparameterization, thus ensuring both accurate quantization and efficient
inference. More specifically, we focus on two components with extreme
distributions: LayerNorm activations and Softmax activations. Initially, we
apply channel-wise quantization and log$\sqrt{2}$ quantization, respectively,
which are tailored to their distributions. In particular, for the former, we
introduce a learnable per-channel dual clipping scheme, which is designed to
efficiently identify outliers in the unbalanced activations with fine
granularity. Then, we reparameterize the scales to hardware-friendly layer-wise
quantization and log2 quantization for inference. Moreover, quantized weight
reconstruction is seamlessly integrated into the above procedure to further
push the performance limits. Extensive experiments are performed on different
large-scale transformer variants on multiple tasks, including vision, language,
and multi-modal transformers, and RepQuant encouragingly demonstrates
significant performance advantages.