Exploring the Rate-Distortion-Complexity Optimization in Neural Image
  Compression
Yixin Gao Runsen Feng Zongyu Guo Zhibo Chen
Despite a short history, neural image codecs have been shown to surpass
classical image codecs in terms of rate-distortion performance. However, most
of them suffer from significantly longer decoding times, which hinders the
practical applications of neural image codecs. This issue is especially
pronounced when employing an effective yet time-consuming autoregressive
context model since it would increase entropy decoding time by orders of
magnitude. In this paper, unlike most previous works that pursue optimal RD
performance while temporally overlooking the coding complexity, we make a
systematical investigation on the rate-distortion-complexity (RDC) optimization
in neural image compression. By quantifying the decoding complexity as a factor
in the optimization goal, we are now able to precisely control the RDC
trade-off and then demonstrate how the rate-distortion performance of neural
image codecs could adapt to various complexity demands. Going beyond the
investigation of RDC optimization, a variable-complexity neural codec is
designed to leverage the spatial dependencies adaptively according to
industrial demands, which supports fine-grained complexity adjustment by
balancing the RDC tradeoff. By implementing this scheme in a powerful base
model, we demonstrate the feasibility and flexibility of RDC optimization for
neural image codecs.