Learning Novel View Synthesis from Heterogeneous Low-light Captures
Quan Zheng Hao Sun Huiyao Xu Fanjiang Xu
Neural radiance field has achieved fundamental success in novel view
synthesis from input views with the same brightness level captured under fixed
normal lighting. Unfortunately, synthesizing novel views remains to be a
challenge for input views with heterogeneous brightness level captured under
low-light condition. The condition is pretty common in the real world. It
causes low-contrast images where details are concealed in the darkness and
camera sensor noise significantly degrades the image quality. To tackle this
problem, we propose to learn to decompose illumination, reflectance, and noise
from input views according to that reflectance remains invariant across
heterogeneous views. To cope with heterogeneous brightness and noise levels
across multi-views, we learn an illumination embedding and optimize a noise map
individually for each view. To allow intuitive editing of the illumination, we
design an illumination adjustment module to enable either brightening or
darkening of the illumination component. Comprehensive experiments demonstrate
that this approach enables effective intrinsic decomposition for low-light
multi-view noisy images and achieves superior visual quality and numerical
performance for synthesizing novel views compared to state-of-the-art methods.