Robustly overfitting latents for flexible neural image compression
Yura Perugachi-Diaz Arwin Gansekoele Sandjai Bhulai
Neural image compression has made a great deal of progress. State-of-the-art
models are based on variational autoencoders and are outperforming classical
models. Neural compression models learn to encode an image into a quantized
latent representation that can be efficiently sent to the decoder, which
decodes the quantized latent into a reconstructed image. While these models
have proven successful in practice, they lead to sub-optimal results due to
imperfect optimization and limitations in the encoder and decoder capacity.
Recent work shows how to use stochastic Gumbel annealing (SGA) to refine the
latents of pre-trained neural image compression models. We extend this idea by
introducing SGA+, which contains three different methods that build upon SGA.
Further, we give a detailed analysis of our proposed methods, show how they
improve performance, and show that they are less sensitive to hyperparameter
choices. Besides, we show how each method can be extended to three- instead of
two-class rounding. Finally, we show how refinement of the latents with our
best-performing method improves the compression performance on the Tecnick
dataset and how it can be deployed to partly move along the rate-distortion
curve.