EGVD: Event-Guided Video Deraining
Yueyi Zhang Jin Wang Wenming Weng Xiaoyan Sun Zhiwei Xiong
With the rapid development of deep learning, video deraining has experienced
significant progress. However, existing video deraining pipelines cannot
achieve satisfying performance for scenes with rain layers of complex
spatio-temporal distribution. In this paper, we approach video deraining by
employing an event camera. As a neuromorphic sensor, the event camera suits
scenes of non-uniform motion and dynamic light conditions. We propose an
end-to-end learning-based network to unlock the potential of the event camera
for video deraining. First, we devise an event-aware motion detection module to
adaptively aggregate multi-frame motion contexts using event-aware masks.
Second, we design a pyramidal adaptive selection module for reliably separating
the background and rain layers by incorporating multi-modal contextualized
priors. In addition, we build a real-world dataset consisting of rainy videos
and temporally synchronized event streams. We compare our method with extensive
state-of-the-art methods on synthetic and self-collected real-world datasets,
demonstrating the clear superiority of our method. The code and dataset are
available at \url{https://github.com/booker-max/EGVD}.