Implicit Learning of Scene Geometry from Poses for Global Localization
Mohammad Altillawi Shile Li Sai Manoj Prakhya Ziyuan Liu Joan Serrat
Global visual localization estimates the absolute pose of a camera using a
single image, in a previously mapped area. Obtaining the pose from a single
image enables many robotics and augmented/virtual reality applications.
Inspired by latest advances in deep learning, many existing approaches directly
learn and regress 6 DoF pose from an input image. However, these methods do not
fully utilize the underlying scene geometry for pose regression. The challenge
in monocular relocalization is the minimal availability of supervised training
data, which is just the corresponding 6 DoF poses of the images. In this paper,
we propose to utilize these minimal available labels (.i.e, poses) to learn the
underlying 3D geometry of the scene and use the geometry to estimate the 6 DoF
camera pose. We present a learning method that uses these pose labels and rigid
alignment to learn two 3D geometric representations (\textit{X, Y, Z
coordinates}) of the scene, one in camera coordinate frame and the other in
global coordinate frame. Given a single image, it estimates these two 3D scene
representations, which are then aligned to estimate a pose that matches the
pose label. This formulation allows for the active inclusion of additional
learning constraints to minimize 3D alignment errors between the two 3D scene
representations, and 2D re-projection errors between the 3D global scene
representation and 2D image pixels, resulting in improved localization
accuracy. During inference, our model estimates the 3D scene geometry in camera
and global frames and aligns them rigidly to obtain pose in real-time. We
evaluate our work on three common visual localization datasets, conduct
ablation studies, and show that our method exceeds state-of-the-art regression
methods' pose accuracy on all datasets.