EVI-SAM: Robust, Real-time, Tightly-coupled Event-Visual-Inertial State
  Estimation and 3D Dense Mapping
Weipeng Guan Peiyu Chen Huibin Zhao Yu Wang Peng Lu
Event cameras are bio-inspired, motion-activated sensors that demonstrate
substantial potential in handling challenging situations, such as motion blur
and high-dynamic range. In this paper, we proposed EVI-SAM to tackle the
problem of 6 DoF pose tracking and 3D reconstruction using monocular event
camera. A novel event-based hybrid tracking framework is designed to estimate
the pose, leveraging the robustness of feature matching and the precision of
direct alignment. Specifically, we develop an event-based 2D-2D alignment to
construct the photometric constraint, and tightly integrate it with the
event-based reprojection constraint. The mapping module recovers the dense and
colorful depth of the scene through the image-guided event-based mapping
method. Subsequently, the appearance, texture, and surface mesh of the 3D scene
can be reconstructed by fusing the dense depth map from multiple viewpoints
using truncated signed distance function (TSDF) fusion. To the best of our
knowledge, this is the first non-learning work to realize event-based dense
mapping. Numerical evaluations are performed on both publicly available and
self-collected datasets, which qualitatively and quantitatively demonstrate the
superior performance of our method. Our EVI-SAM effectively balances accuracy
and robustness while maintaining computational efficiency, showcasing superior
pose tracking and dense mapping performance in challenging scenarios. Video
Demo: https://youtu.be/Nn40U4e5Si8.