MM-Gaussian: 3D Gaussian-based Multi-modal Fusion for Localization and
  Reconstruction in Unbounded Scenes
Chenyang Wu Yifan Duan Xinran Zhang Yu Sheng Jianmin Ji Yanyong Zhang
Localization and mapping are critical tasks for various applications such as
autonomous vehicles and robotics. The challenges posed by outdoor environments
present particular complexities due to their unbounded characteristics. In this
work, we present MM-Gaussian, a LiDAR-camera multi-modal fusion system for
localization and mapping in unbounded scenes. Our approach is inspired by the
recently developed 3D Gaussians, which demonstrate remarkable capabilities in
achieving high rendering quality and fast rendering speed. Specifically, our
system fully utilizes the geometric structure information provided by
solid-state LiDAR to address the problem of inaccurate depth encountered when
relying solely on visual solutions in unbounded, outdoor scenarios.
Additionally, we utilize 3D Gaussian point clouds, with the assistance of
pixel-level gradient descent, to fully exploit the color information in photos,
thereby achieving realistic rendering effects. To further bolster the
robustness of our system, we designed a relocalization module, which assists in
returning to the correct trajectory in the event of a localization failure.
Experiments conducted in multiple scenarios demonstrate the effectiveness of
our method.