Event Cameras Meet SPADs for High-Speed, Low-Bandwidth Imaging
Manasi Muglikar Siddharth Somasundaram Akshat Dave Edoardo Charbon Ramesh Raskar Davide Scaramuzza
Traditional cameras face a trade-off between low-light performance and
high-speed imaging: longer exposure times to capture sufficient light results
in motion blur, whereas shorter exposures result in Poisson-corrupted noisy
images. While burst photography techniques help mitigate this tradeoff,
conventional cameras are fundamentally limited in their sensor noise
characteristics. Event cameras and single-photon avalanche diode (SPAD) sensors
have emerged as promising alternatives to conventional cameras due to their
desirable properties. SPADs are capable of single-photon sensitivity with
microsecond temporal resolution, and event cameras can measure brightness
changes up to 1 MHz with low bandwidth requirements. We show that these
properties are complementary, and can help achieve low-light, high-speed image
reconstruction with low bandwidth requirements. We introduce a sensor fusion
framework to combine SPADs with event cameras to improves the reconstruction of
high-speed, low-light scenes while reducing the high bandwidth cost associated
with using every SPAD frame. Our evaluation, on both synthetic and real sensor
data, demonstrates significant enhancements ( > 5 dB PSNR) in reconstructing
low-light scenes at high temporal resolution (100 kHz) compared to conventional
cameras. Event-SPAD fusion shows great promise for real-world applications,
such as robotics or medical imaging.