VoxelNextFusion: A Simple, Unified and Effective Voxel Fusion Framework
  for Multi-Modal 3D Object Detection
Ziying Song Guoxin Zhang Jun Xie Lin Liu Caiyan Jia Shaoqing Xu Zhepeng Wang
LiDAR-camera fusion can enhance the performance of 3D object detection by
utilizing complementary information between depth-aware LiDAR points and
semantically rich images. Existing voxel-based methods face significant
challenges when fusing sparse voxel features with dense image features in a
one-to-one manner, resulting in the loss of the advantages of images, including
semantic and continuity information, leading to sub-optimal detection
performance, especially at long distances. In this paper, we present
VoxelNextFusion, a multi-modal 3D object detection framework specifically
designed for voxel-based methods, which effectively bridges the gap between
sparse point clouds and dense images. In particular, we propose a voxel-based
image pipeline that involves projecting point clouds onto images to obtain both
pixel- and patch-level features. These features are then fused using a
self-attention to obtain a combined representation. Moreover, to address the
issue of background features present in patches, we propose a feature
importance module that effectively distinguishes between foreground and
background features, thus minimizing the impact of the background features.
Extensive experiments were conducted on the widely used KITTI and nuScenes 3D
object detection benchmarks. Notably, our VoxelNextFusion achieved around
+3.20% in AP@0.7 improvement for car detection in hard level compared to the
Voxel R-CNN baseline on the KITTI test dataset