Effortless Cross-Platform Video Codec: A Codebook-Based Method
Kuan Tian Yonghang Guan Jinxi Xiang Jun Zhang Xiao Han Wei Yang
Under certain circumstances, advanced neural video codecs can surpass the
most complex traditional codecs in their rate-distortion (RD) performance. One
of the main reasons for the high performance of existing neural video codecs is
the use of the entropy model, which can provide more accurate probability
distribution estimations for compressing the latents. This also implies the
rigorous requirement that entropy models running on different platforms should
use consistent distribution estimations. However, in cross-platform scenarios,
entropy models running on different platforms usually yield inconsistent
probability distribution estimations due to floating point computation errors
that are platform-dependent, which can cause the decoding side to fail in
correctly decoding the compressed bitstream sent by the encoding side. In this
paper, we propose a cross-platform video compression framework based on
codebooks, which avoids autoregressive entropy modeling and achieves video
compression by transmitting the index sequence of the codebooks. Moreover,
instead of using optical flow for context alignment, we propose to use the
conditional cross-attention module to obtain the context between frames. Due to
the absence of autoregressive modeling and optical flow alignment, we can
design an extremely minimalist framework that can greatly benefit computational
efficiency. Importantly, our framework no longer contains any distribution
estimation modules for entropy modeling, and thus computations across platforms
are not necessarily consistent. Experimental results show that our method can
outperform the traditional H.265 (medium) even without any entropy constraints,
while achieving the cross-platform property intrinsically.