Temporal Consistent Automatic Video Colorization via Semantic
  Correspondence
Yu Zhang Siqi Chen Mingdao Wang Xianlin Zhang Chuang Zhu Yue Zhang Xueming Li
Video colorization task has recently attracted wide attention. Recent methods
mainly work on the temporal consistency in adjacent frames or frames with small
interval. However, it still faces severe challenge of the inconsistency between
frames with large interval.To address this issue, we propose a novel video
colorization framework, which combines semantic correspondence into automatic
video colorization to keep long-range consistency. Firstly, a reference
colorization network is designed to automatically colorize the first frame of
each video, obtaining a reference image to supervise the following whole
colorization process. Such automatically colorized reference image can not only
avoid labor-intensive and time-consuming manual selection, but also enhance the
similarity between reference and grayscale images. Afterwards, a semantic
correspondence network and an image colorization network are introduced to
colorize a series of the remaining frames with the help of the reference. Each
frame is supervised by both the reference image and the immediately colorized
preceding frame to improve both short-range and long-range temporal
consistency. Extensive experiments demonstrate that our method outperforms
other methods in maintaining temporal consistency both qualitatively and
quantitatively. In the NTIRE 2023 Video Colorization Challenge, our method
ranks at the 3rd place in Color Distribution Consistency (CDC) Optimization
track.