DVLO: Deep Visual-LiDAR Odometry with Local-to-Global Feature Fusion and
  Bi-Directional Structure Alignment
Jiuming Liu Dong Zhuo Zhiheng Feng Siting Zhu Chensheng Peng Zhe Liu Hesheng Wang
Information inside visual and LiDAR data is well complementary derived from
the fine-grained texture of images and massive geometric information in point
clouds. However, it remains challenging to explore effective visual-LiDAR
fusion, mainly due to the intrinsic data structure inconsistency between two
modalities: Images are regular and dense, but LiDAR points are unordered and
sparse. To address the problem, we propose a local-to-global fusion network
with bi-directional structure alignment. To obtain locally fused features, we
project points onto image plane as cluster centers and cluster image pixels
around each center. Image pixels are pre-organized as pseudo points for
image-to-point structure alignment. Then, we convert points to pseudo images by
cylindrical projection (point-to-image structure alignment) and perform
adaptive global feature fusion between point features with local fused
features. Our method achieves state-of-the-art performance on KITTI odometry
and FlyingThings3D scene flow datasets compared to both single-modal and
multi-modal methods. Codes will be released later.