Real-time Noise Source Estimation of a Camera System from an Image and
  Metadata
Maik Wischow Patrick Irmisch Anko Boerner Guillermo Gallego
Autonomous machines must self-maintain proper functionality to ensure the
safety of humans and themselves. This pertains particularly to its cameras as
predominant sensors to perceive the environment and support actions. A
fundamental camera problem addressed in this study is noise. Solutions often
focus on denoising images a posteriori, that is, fighting symptoms rather than
root causes. However, tackling root causes requires identifying the noise
sources, considering the limitations of mobile platforms. This work
investigates a real-time, memory-efficient and reliable noise source estimator
that combines data- and physically-based models. To this end, a DNN that
examines an image with camera metadata for major camera noise sources is built
and trained. In addition, it quantifies unexpected factors that impact image
noise or metadata. This study investigates seven different estimators on six
datasets that include synthetic noise, real-world noise from two camera
systems, and real field campaigns. For these, only the model with most metadata
is capable to accurately and robustly quantify all individual noise
contributions. This method outperforms total image noise estimators and can be
plug-and-play deployed. It also serves as a basis to include more advanced
noise sources, or as part of an automatic countermeasure feedback-loop to
approach fully reliable machines.