NeSLAM: Neural Implicit Mapping and Self-Supervised Feature Tracking
  With Depth Completion and Denoising
Tianchen Deng Yanbo Wang Hongle Xie Hesheng Wang Jingchuan Wang Danwei Wang Weidong Chen
In recent years, there have been significant advancements in 3D
reconstruction and dense RGB-D SLAM systems. One notable development is the
application of Neural Radiance Fields (NeRF) in these systems, which utilizes
implicit neural representation to encode 3D scenes. This extension of NeRF to
SLAM has shown promising results. However, the depth images obtained from
consumer-grade RGB-D sensors are often sparse and noisy, which poses
significant challenges for 3D reconstruction and affects the accuracy of the
representation of the scene geometry. Moreover, the original hierarchical
feature grid with occupancy value is inaccurate for scene geometry
representation. Furthermore, the existing methods select random pixels for
camera tracking, which leads to inaccurate localization and is not robust in
real-world indoor environments. To this end, we present NeSLAM, an advanced
framework that achieves accurate and dense depth estimation, robust camera
tracking, and realistic synthesis of novel views. First, a depth completion and
denoising network is designed to provide dense geometry prior and guide the
neural implicit representation optimization. Second, the occupancy scene
representation is replaced with Signed Distance Field (SDF) hierarchical scene
representation for high-quality reconstruction and view synthesis. Furthermore,
we also propose a NeRF-based self-supervised feature tracking algorithm for
robust real-time tracking. Experiments on various indoor datasets demonstrate
the effectiveness and accuracy of the system in reconstruction, tracking
quality, and novel view synthesis.