GraphAlign: Enhancing Accurate Feature Alignment by Graph matching for
  Multi-Modal 3D Object Detection
Ziying Song Haiyue Wei Lin Bai Lei Yang Caiyan Jia
LiDAR and cameras are complementary sensors for 3D object detection in
autonomous driving. However, it is challenging to explore the unnatural
interaction between point clouds and images, and the critical factor is how to
conduct feature alignment of heterogeneous modalities. Currently, many methods
achieve feature alignment by projection calibration only, without considering
the problem of coordinate conversion accuracy errors between sensors, leading
to sub-optimal performance. In this paper, we present GraphAlign, a more
accurate feature alignment strategy for 3D object detection by graph matching.
Specifically, we fuse image features from a semantic segmentation encoder in
the image branch and point cloud features from a 3D Sparse CNN in the LiDAR
branch. To save computation, we construct the nearest neighbor relationship by
calculating Euclidean distance within the subspaces that are divided into the
point cloud features. Through the projection calibration between the image and
point cloud, we project the nearest neighbors of point cloud features onto the
image features. Then by matching the nearest neighbors with a single point
cloud to multiple images, we search for a more appropriate feature alignment.
In addition, we provide a self-attention module to enhance the weights of
significant relations to fine-tune the feature alignment between heterogeneous
modalities. Extensive experiments on nuScenes benchmark demonstrate the
effectiveness and efficiency of our GraphAlign.