Leveraging Neural Radiance Fields for Uncertainty-Aware Visual
  Localization
Le Chen Weirong Chen Rui Wang Marc Pollefeys
As a promising fashion for visual localization, scene coordinate regression
(SCR) has seen tremendous progress in the past decade. Most recent methods
usually adopt neural networks to learn the mapping from image pixels to 3D
scene coordinates, which requires a vast amount of annotated training data. We
propose to leverage Neural Radiance Fields (NeRF) to generate training samples
for SCR. Despite NeRF's efficiency in rendering, many of the rendered data are
polluted by artifacts or only contain minimal information gain, which can
hinder the regression accuracy or bring unnecessary computational costs with
redundant data. These challenges are addressed in three folds in this paper:
(1) A NeRF is designed to separately predict uncertainties for the rendered
color and depth images, which reveal data reliability at the pixel level. (2)
SCR is formulated as deep evidential learning with epistemic uncertainty, which
is used to evaluate information gain and scene coordinate quality. (3) Based on
the three arts of uncertainties, a novel view selection policy is formed that
significantly improves data efficiency. Experiments on public datasets
demonstrate that our method could select the samples that bring the most
information gain and promote the performance with the highest efficiency.