On Uniform Scalar Quantization for Learned Image Compression
Haotian Zhang Li Li Dong Liu
Learned image compression possesses a unique challenge when incorporating
non-differentiable quantization into the gradient-based training of the
networks. Several quantization surrogates have been proposed to fulfill the
training, but they were not systematically justified from a theoretical
perspective. We fill this gap by contrasting uniform scalar quantization, the
most widely used category with rounding being its simplest case, and its
training surrogates. In principle, we find two factors crucial: one is the
discrepancy between the surrogate and rounding, leading to train-test mismatch;
the other is gradient estimation risk due to the surrogate, which consists of
bias and variance of the gradient estimation. Our analyses and simulations
imply that there is a tradeoff between the train-test mismatch and the gradient
estimation risk, and the tradeoff varies across different network structures.
Motivated by these analyses, we present a method based on stochastic uniform
annealing, which has an adjustable temperature coefficient to control the
tradeoff. Moreover, our analyses enlighten us as to two subtle tricks: one is
to set an appropriate lower bound for the variance parameter of the estimated
quantized latent distribution, which effectively reduces the train-test
mismatch; the other is to use zero-center quantization with partial
stop-gradient, which reduces the gradient estimation variance and thus
stabilize the training. Our method with the tricks is verified to outperform
the existing practices of quantization surrogates on a variety of
representative image compression networks.