SUNDIAL: 3D Satellite Understanding through Direct, Ambient, and Complex
  Lighting Decomposition
Nikhil Behari Akshat Dave Kushagra Tiwary William Yang Ramesh Raskar
3D modeling from satellite imagery is essential in areas of environmental
science, urban planning, agriculture, and disaster response. However,
traditional 3D modeling techniques face unique challenges in the remote sensing
context, including limited multi-view baselines over extensive regions, varying
direct, ambient, and complex illumination conditions, and time-varying scene
changes across captures. In this work, we introduce SUNDIAL, a comprehensive
approach to 3D reconstruction of satellite imagery using neural radiance
fields. We jointly learn satellite scene geometry, illumination components, and
sun direction in this single-model approach, and propose a secondary shadow ray
casting technique to 1) improve scene geometry using oblique sun angles to
render shadows, 2) enable physically-based disentanglement of scene albedo and
illumination, and 3) determine the components of illumination from direct,
ambient (sky), and complex sources. To achieve this, we incorporate lighting
cues and geometric priors from remote sensing literature in a neural rendering
approach, modeling physical properties of satellite scenes such as shadows,
scattered sky illumination, and complex illumination and shading of vegetation
and water. We evaluate the performance of SUNDIAL against existing NeRF-based
techniques for satellite scene modeling and demonstrate improved scene and
lighting disentanglement, novel view and lighting rendering, and geometry and
sun direction estimation on challenging scenes with small baselines, sparse
inputs, and variable illumination.