Learning Parallax for Stereo Event-based Motion Deblurring
Mingyuan Lin Chi Zhang Chu He Lei Yu
Due to the extremely low latency, events have been recently exploited to
supplement lost information for motion deblurring. Existing approaches largely
rely on the perfect pixel-wise alignment between intensity images and events,
which is not always fulfilled in the real world. To tackle this problem, we
propose a novel coarse-to-fine framework, named NETwork of Event-based motion
Deblurring with STereo event and intensity cameras (St-EDNet), to recover
high-quality images directly from the misaligned inputs, consisting of a single
blurry image and the concurrent event streams. Specifically, the coarse spatial
alignment of the blurry image and the event streams is first implemented with a
cross-modal stereo matching module without the need for ground-truth depths.
Then, a dual-feature embedding architecture is proposed to gradually build the
fine bidirectional association of the coarsely aligned data and reconstruct the
sequence of the latent sharp images. Furthermore, we build a new dataset with
STereo Event and Intensity Cameras (StEIC), containing real-world events,
intensity images, and dense disparity maps. Experiments on real-world datasets
demonstrate the superiority of the proposed network over state-of-the-art
methods.