Neural Image Re-Exposure
Xinyu Zhang Hefei Huang Xu Jia Dong Wang Huchuan Lu
The shutter strategy applied to the photo-shooting process has a significant
influence on the quality of the captured photograph. An improper shutter may
lead to a blurry image, video discontinuity, or rolling shutter artifact.
Existing works try to provide an independent solution for each issue. In this
work, we aim to re-expose the captured photo in post-processing to provide a
more flexible way of addressing those issues within a unified framework.
Specifically, we propose a neural network-based image re-exposure framework. It
consists of an encoder for visual latent space construction, a re-exposure
module for aggregating information to neural film with a desired shutter
strategy, and a decoder for 'developing' neural film into a desired image. To
compensate for information confusion and missing frames, event streams, which
can capture almost continuous brightness changes, are leveraged in computing
visual latent content. Both self-attention layers and cross-attention layers
are employed in the re-exposure module to promote interaction between neural
film and visual latent content and information aggregation to neural film. The
proposed unified image re-exposure framework is evaluated on several
shutter-related image recovery tasks and performs favorably against independent
state-of-the-art methods.