Bridging the Gap: Fine-to-Coarse Sketch Interpolation Network for
  High-Quality Animation Sketch Inbetweening
Jiaming Shen Kun Hu Wei Bao Chang Wen Chen Zhiyong Wang
The 2D animation workflow is typically initiated with the creation of
keyframes using sketch-based drawing. Subsequent inbetweens (i.e., intermediate
sketch frames) are crafted through manual interpolation for smooth animations,
which is a labor-intensive process. Thus, the prospect of automatic animation
sketch interpolation has become highly appealing. However, existing video
interpolation methods are generally hindered by two key issues for sketch
inbetweening: 1) limited texture and colour details in sketches, and 2)
exaggerated alterations between two sketch keyframes. To overcome these issues,
we propose a novel deep learning method, namely Fine-to-Coarse Sketch
Interpolation Network (FC-SIN). This approach incorporates multi-level guidance
that formulates region-level correspondence, sketch-level correspondence and
pixel-level dynamics. A multi-stream U-Transformer is then devised to
characterize sketch inbewteening patterns using these multi-level guides
through the integration of both self-attention and cross-attention mechanisms.
Additionally, to facilitate future research on animation sketch inbetweening,
we constructed a large-scale dataset - STD-12K, comprising 30 sketch animation
series in diverse artistic styles. Comprehensive experiments on this dataset
convincingly show that our proposed FC-SIN surpasses the state-of-the-art
interpolation methods. Our code and dataset will be publicly available.