DNS SLAM: Dense Neural Semantic-Informed SLAM
Kunyi Li Michael Niemeyer Nassir Navab Federico Tombari
In recent years, coordinate-based neural implicit representations have shown
promising results for the task of Simultaneous Localization and Mapping (SLAM).
While achieving impressive performance on small synthetic scenes, these methods
often suffer from oversmoothed reconstructions, especially for complex
real-world scenes. In this work, we introduce DNS SLAM, a novel neural RGB-D
semantic SLAM approach featuring a hybrid representation. Relying only on 2D
semantic priors, we propose the first semantic neural SLAM method that trains
class-wise scene representations while providing stable camera tracking at the
same time. Our method integrates multi-view geometry constraints with
image-based feature extraction to improve appearance details and to output
color, density, and semantic class information, enabling many downstream
applications. To further enable real-time tracking, we introduce a lightweight
coarse scene representation which is trained in a self-supervised manner in
latent space. Our experimental results achieve state-of-the-art performance on
both synthetic data and real-world data tracking while maintaining a
commendable operational speed on off-the-shelf hardware. Further, our method
outputs class-wise decomposed reconstructions with better texture capturing
appearance and geometric details.