DBA-Fusion: Tightly Integrating Deep Dense Visual Bundle Adjustment with
  Multiple Sensors for Large-Scale Localization and Mapping
Yuxuan Zhou Xingxing Li Shengyu Li Xuanbin Wang Shaoquan Feng Yuxuan Tan
Visual simultaneous localization and mapping (VSLAM) has broad applications,
with state-of-the-art methods leveraging deep neural networks for better
robustness and applicability. However, there is a lack of research in fusing
these learning-based methods with multi-sensor information, which could be
indispensable to push related applications to large-scale and complex
scenarios. In this paper, we tightly integrate the trainable deep dense bundle
adjustment (DBA) with multi-sensor information through a factor graph. In the
framework, recurrent optical flow and DBA are performed among sequential
images. The Hessian information derived from DBA is fed into a generic factor
graph for multi-sensor fusion, which employs a sliding window and supports
probabilistic marginalization. A pipeline for visual-inertial integration is
firstly developed, which provides the minimum ability of metric-scale
localization and mapping. Furthermore, other sensors (e.g., global navigation
satellite system) are integrated for driftless and geo-referencing
functionality. Extensive tests are conducted on both public datasets and
self-collected datasets. The results validate the superior localization
performance of our approach, which enables real-time dense mapping in
large-scale environments. The code has been made open-source
(https://github.com/GREAT-WHU/DBA-Fusion).