EventAid: Benchmarking Event-aided Image/Video Enhancement Algorithms
  with Real-captured Hybrid Dataset
Peiqi Duan Boyu Li Yixin Yang Hanyue Lou Minggui Teng Yi Ma Boxin Shi
Event cameras are emerging imaging technology that offers advantages over
conventional frame-based imaging sensors in dynamic range and sensing speed.
Complementing the rich texture and color perception of traditional image
frames, the hybrid camera system of event and frame-based cameras enables
high-performance imaging. With the assistance of event cameras, high-quality
image/video enhancement methods make it possible to break the limits of
traditional frame-based cameras, especially exposure time, resolution, dynamic
range, and frame rate limits. This paper focuses on five event-aided image and
video enhancement tasks (i.e., event-based video reconstruction, event-aided
high frame rate video reconstruction, image deblurring, image super-resolution,
and high dynamic range image reconstruction), provides an analysis of the
effects of different event properties, a real-captured and ground truth labeled
benchmark dataset, a unified benchmarking of state-of-the-art methods, and an
evaluation for two mainstream event simulators. In detail, this paper collects
a real-captured evaluation dataset EventAid for five event-aided image/video
enhancement tasks, by using "Event-RGB" multi-camera hybrid system, taking into
account scene diversity and spatiotemporal synchronization. We further perform
quantitative and visual comparisons for state-of-the-art algorithms, provide a
controlled experiment to analyze the performance limit of event-aided image
deblurring methods, and discuss open problems to inspire future research.