Three-Stage Cascade Framework for Blurry Video Frame Interpolation
Pengcheng Lei Zaoming Yan Tingting Wang Faming Fang Guixu Zhang
Blurry video frame interpolation (BVFI) aims to generate high-frame-rate
clear videos from low-frame-rate blurry videos, is a challenging but important
topic in the computer vision community. Blurry videos not only provide spatial
and temporal information like clear videos, but also contain additional motion
information hidden in each blurry frame. However, existing BVFI methods usually
fail to fully leverage all valuable information, which ultimately hinders their
performance. In this paper, we propose a simple end-to-end three-stage
framework to fully explore useful information from blurry videos. The frame
interpolation stage designs a temporal deformable network to directly sample
useful information from blurry inputs and synthesize an intermediate frame at
an arbitrary time interval. The temporal feature fusion stage explores the
long-term temporal information for each target frame through a bi-directional
recurrent deformable alignment network. And the deblurring stage applies a
transformer-empowered Taylor approximation network to recursively recover the
high-frequency details. The proposed three-stage framework has clear task
assignment for each module and offers good expandability, the effectiveness of
which are demonstrated by various experimental results. We evaluate our model
on four benchmarks, including the Adobe240 dataset, GoPro dataset, YouTube240
dataset and Sony dataset. Quantitative and qualitative results indicate that
our model outperforms existing SOTA methods. Besides, experiments on real-world
blurry videos also indicate the good generalization ability of our model.