Compute-first optical detection for noise-resilient visual perception
Jungmin Kim Nanfang Yu Zongfu Yu
In the context of visual perception, the optical signal from a scene is
transferred into the electronic domain by detectors in the form of image data,
which are then processed for the extraction of visual information. In noisy and
weak-signal environments such as thermal imaging for night vision applications,
however, the performance of neural computing tasks faces a significant
bottleneck due to the inherent degradation of data quality upon noisy
detection. Here, we propose a concept of optical signal processing before
detection to address this issue. We demonstrate that spatially redistributing
optical signals through a properly designed linear transformer can enhance the
detection noise resilience of visual perception tasks, as benchmarked with the
MNIST classification. Our idea is supported by a quantitative analysis
detailing the relationship between signal concentration and noise robustness,
as well as its practical implementation in an incoherent imaging system. This
compute-first detection scheme can pave the way for advancing infrared machine
vision technologies widely used for industrial and defense applications.