Aggregating Long-term Sharp Features via Hybrid Transformers for Video
  Deblurring
Dongwei Ren Wei Shang Yi Yang Wangmeng Zuo
Video deblurring methods, aiming at recovering consecutive sharp frames from
a given blurry video, usually assume that the input video suffers from
consecutively blurry frames. However, in real-world blurry videos taken by
modern imaging devices, sharp frames usually appear in the given video, thus
making temporal long-term sharp features available for facilitating the
restoration of a blurry frame. In this work, we propose a video deblurring
method that leverages both neighboring frames and present sharp frames using
hybrid Transformers for feature aggregation. Specifically, we first train a
blur-aware detector to distinguish between sharp and blurry frames. Then, a
window-based local Transformer is employed for exploiting features from
neighboring frames, where cross attention is beneficial for aggregating
features from neighboring frames without explicit spatial alignment. To
aggregate long-term sharp features from detected sharp frames, we utilize a
global Transformer with multi-scale matching capability. Moreover, our method
can easily be extended to event-driven video deblurring by incorporating an
event fusion module into the global Transformer. Extensive experiments on
benchmark datasets demonstrate that our proposed method outperforms
state-of-the-art video deblurring methods as well as event-driven video
deblurring methods in terms of quantitative metrics and visual quality. The
source code and trained models are available at
https://github.com/shangwei5/STGTN.