Perception-Oriented Video Frame Interpolation via Asymmetric Blending
Guangyang Wu Xin Tao Changlin Li Wenyi Wang Xiaohong Liu Qingqing Zheng
Previous methods for Video Frame Interpolation (VFI) have encountered
challenges, notably the manifestation of blur and ghosting effects. These
issues can be traced back to two pivotal factors: unavoidable motion errors and
misalignment in supervision. In practice, motion estimates often prove to be
error-prone, resulting in misaligned features. Furthermore, the reconstruction
loss tends to bring blurry results, particularly in misaligned regions. To
mitigate these challenges, we propose a new paradigm called PerVFI
(Perception-oriented Video Frame Interpolation). Our approach incorporates an
Asymmetric Synergistic Blending module (ASB) that utilizes features from both
sides to synergistically blend intermediate features. One reference frame
emphasizes primary content, while the other contributes complementary
information. To impose a stringent constraint on the blending process, we
introduce a self-learned sparse quasi-binary mask which effectively mitigates
ghosting and blur artifacts in the output. Additionally, we employ a
normalizing flow-based generator and utilize the negative log-likelihood loss
to learn the conditional distribution of the output, which further facilitates
the generation of clear and fine details. Experimental results validate the
superiority of PerVFI, demonstrating significant improvements in perceptual
quality compared to existing methods. Codes are available at
\url{https://github.com/mulns/PerVFI}