Density Uncertainty Quantification with NeRF-Ensembles: Impact of Data
  and Scene Constraints
Miriam Jäger Steven Landgraf Boris Jutzi
In the fields of computer graphics, computer vision and photogrammetry,
Neural Radiance Fields (NeRFs) are a major topic driving current research and
development. However, the quality of NeRF-generated 3D scene reconstructions
and subsequent surface reconstructions, heavily relies on the network output,
particularly the density. Regarding this critical aspect, we propose to utilize
NeRF-Ensembles that provide a density uncertainty estimate alongside the mean
density. We demonstrate that data constraints such as low-quality images and
poses lead to a degradation of the training process, increased density
uncertainty and decreased predicted density. Even with high-quality input data,
the density uncertainty varies based on scene constraints such as acquisition
constellations, occlusions and material properties. NeRF-Ensembles not only
provide a tool for quantifying the uncertainty but exhibit two promising
advantages: Enhanced robustness and artifact removal. Through the utilization
of NeRF-Ensembles instead of single NeRFs, small outliers are removed, yielding
a smoother output with improved completeness of structures. Furthermore,
applying percentile-based thresholds on density uncertainty outliers proves to
be effective for the removal of large (foggy) artifacts in post-processing. We
conduct our methodology on 3 different datasets: (i) synthetic benchmark
dataset, (ii) real benchmark dataset, (iii) real data under realistic recording
conditions and sensors.