Noise2Image: Noise-Enabled Static Scene Recovery for Event Cameras
Ruiming Cao Dekel Galor Amit Kohli Jacob L Yates Laura Waller
Event cameras capture changes of intensity over time as a stream of 'events'
and generally cannot measure intensity itself; hence, they are only used for
imaging dynamic scenes. However, fluctuations due to random photon arrival
inevitably trigger noise events, even for static scenes. While previous efforts
have been focused on filtering out these undesirable noise events to improve
signal quality, we find that, in the photon-noise regime, these noise events
are correlated with the static scene intensity. We analyze the noise event
generation and model its relationship to illuminance. Based on this
understanding, we propose a method, called Noise2Image, to leverage the
illuminance-dependent noise characteristics to recover the static parts of a
scene, which are otherwise invisible to event cameras. We experimentally
collect a dataset of noise events on static scenes to train and validate
Noise2Image. Our results show that Noise2Image can robustly recover intensity
images solely from noise events, providing a novel approach for capturing
static scenes in event cameras, without additional hardware.