VRS-NeRF: Visual Relocalization with Sparse Neural Radiance Field
Fei Xue Ignas Budvytis Daniel Olmeda Reino Roberto Cipolla
Visual relocalization is a key technique to autonomous driving, robotics, and
virtual/augmented reality. After decades of explorations, absolute pose
regression (APR), scene coordinate regression (SCR), and hierarchical methods
(HMs) have become the most popular frameworks. However, in spite of high
efficiency, APRs and SCRs have limited accuracy especially in large-scale
outdoor scenes; HMs are accurate but need to store a large number of 2D
descriptors for matching, resulting in poor efficiency. In this paper, we
propose an efficient and accurate framework, called VRS-NeRF, for visual
relocalization with sparse neural radiance field. Precisely, we introduce an
explicit geometric map (EGM) for 3D map representation and an implicit learning
map (ILM) for sparse patches rendering. In this localization process, EGP
provides priors of spare 2D points and ILM utilizes these sparse points to
render patches with sparse NeRFs for matching. This allows us to discard a
large number of 2D descriptors so as to reduce the map size. Moreover,
rendering patches only for useful points rather than all pixels in the whole
image reduces the rendering time significantly. This framework inherits the
accuracy of HMs and discards their low efficiency. Experiments on 7Scenes,
CambridgeLandmarks, and Aachen datasets show that our method gives much better
accuracy than APRs and SCRs, and close performance to HMs but is much more
efficient.