VioLA: Aligning Videos to 2D LiDAR Scans
Jun-Jee Chao Selim Engin Nikhil Chavan-Dafle Bhoram Lee Volkan Isler
We study the problem of aligning a video that captures a local portion of an
environment to the 2D LiDAR scan of the entire environment. We introduce a
method (VioLA) that starts with building a semantic map of the local scene from
the image sequence, then extracts points at a fixed height for registering to
the LiDAR map. Due to reconstruction errors or partial coverage of the camera
scan, the reconstructed semantic map may not contain sufficient information for
registration. To address this problem, VioLA makes use of a pre-trained
text-to-image inpainting model paired with a depth completion model for filling
in the missing scene content in a geometrically consistent fashion to support
pose registration. We evaluate VioLA on two real-world RGB-D benchmarks, as
well as a self-captured dataset of a large office scene. Notably, our proposed
scene completion module improves the pose registration performance by up to
20%.