Multiscale Motion-Aware and Spatial-Temporal-Channel Contextual Coding
  Network for Learned Video Compression
Yiming Wang Qian Huang Bin Tang Huashan Sun Xing Li
Recently, learned video compression has achieved exciting performance.
Following the traditional hybrid prediction coding framework, most learned
methods generally adopt the motion estimation motion compensation (MEMC) method
to remove inter-frame redundancy. However, inaccurate motion vector (MV)
usually lead to the distortion of reconstructed frame. In addition, most
approaches ignore the spatial and channel redundancy. To solve above problems,
we propose a motion-aware and spatial-temporal-channel contextual coding based
video compression network (MASTC-VC), which learns the latent representation
and uses variational autoencoders (VAEs) to capture the characteristics of
intra-frame pixels and inter-frame motion. Specifically, we design a multiscale
motion-aware module (MS-MAM) to estimate spatial-temporal-channel consistent
motion vector by utilizing the multiscale motion prediction information in a
coarse-to-fine way. On the top of it, we further propose a
spatial-temporal-channel contextual module (STCCM), which explores the
correlation of latent representation to reduce the bit consumption from
spatial, temporal and channel aspects respectively. Comprehensive experiments
show that our proposed MASTC-VC is surprior to previous state-of-the-art (SOTA)
methods on three public benchmark datasets. More specifically, our method
brings average 10.15\% BD-rate savings against H.265/HEVC (HM-16.20) in PSNR
metric and average 23.93\% BD-rate savings against H.266/VVC (VTM-13.2) in
MS-SSIM metric.