HDR Imaging for Dynamic Scenes with Events
Li Xiaopeng Zeng Zhaoyuan Fan Cien Zhao Chen Deng Lei Yu Lei
High dynamic range imaging (HDRI) for real-world dynamic scenes is
challenging because moving objects may lead to hybrid degradation of low
dynamic range and motion blur. Existing event-based approaches only focus on a
separate task, while cascading HDRI and motion deblurring would lead to
sub-optimal solutions, and unavailable ground-truth sharp HDR images aggravate
the predicament. To address these challenges, we propose an Event-based HDRI
framework within a Self-supervised learning paradigm, i.e., Self-EHDRI, which
generalizes HDRI performance in real-world dynamic scenarios. Specifically, a
self-supervised learning strategy is carried out by learning cross-domain
conversions from blurry LDR images to sharp LDR images, which enables sharp HDR
images to be accessible in the intermediate process even though ground-truth
sharp HDR images are missing. Then, we formulate the event-based HDRI and
motion deblurring model and conduct a unified network to recover the
intermediate sharp HDR results, where both the high dynamic range and high
temporal resolution of events are leveraged simultaneously for compensation. We
construct large-scale synthetic and real-world datasets to evaluate the
effectiveness of our method. Comprehensive experiments demonstrate that the
proposed Self-EHDRI outperforms state-of-the-art approaches by a large margin.
The codes, datasets, and results are available at
https://lxp-whu.github.io/Self-EHDRI.