Toward Accurate and Temporally Consistent Video Restoration from Raw
  Data
Shi Guo Jianqi Ma Xi Yang Zhengqiang Zhang Lei Zhang
Denoising and demosaicking are two fundamental steps in reconstructing a
clean full-color video from raw data, while performing video denoising and
demosaicking jointly, namely VJDD, could lead to better video restoration
performance than performing them separately. In addition to restoration
accuracy, another key challenge to VJDD lies in the temporal consistency of
consecutive frames. This issue exacerbates when perceptual regularization terms
are introduced to enhance video perceptual quality. To address these
challenges, we present a new VJDD framework by consistent and accurate latent
space propagation, which leverages the estimation of previous frames as prior
knowledge to ensure consistent recovery of the current frame. A data temporal
consistency (DTC) loss and a relational perception consistency (RPC) loss are
accordingly designed. Compared with the commonly used flow-based losses, the
proposed losses can circumvent the error accumulation problem caused by
inaccurate flow estimation and effectively handle intensity changes in videos,
improving much the temporal consistency of output videos while preserving
texture details. Extensive experiments demonstrate the leading VJDD performance
of our method in term of restoration accuracy, perceptual quality and temporal
consistency. Codes and dataset are available at
\url{https://github.com/GuoShi28/VJDD}.