Language-Guided 3D Object Detection in Point Cloud for Autonomous
  Driving
Wenhao Cheng Junbo Yin Wei Li Ruigang Yang Jianbing Shen
This paper addresses the problem of 3D referring expression comprehension
(REC) in autonomous driving scenario, which aims to ground a natural language
to the targeted region in LiDAR point clouds. Previous approaches for REC
usually focus on the 2D or 3D-indoor domain, which is not suitable for
accurately predicting the location of the queried 3D region in an autonomous
driving scene. In addition, the upper-bound limitation and the heavy
computation cost motivate us to explore a better solution. In this work, we
propose a new multi-modal visual grounding task, termed LiDAR Grounding. Then
we devise a Multi-modal Single Shot Grounding (MSSG) approach with an effective
token fusion strategy. It jointly learns the LiDAR-based object detector with
the language features and predicts the targeted region directly from the
detector without any post-processing. Moreover, the image feature can be
flexibly integrated into our approach to provide rich texture and color
information. The cross-modal learning enforces the detector to concentrate on
important regions in the point cloud by considering the informative language
expressions, thus leading to much better accuracy and efficiency. Extensive
experiments on the Talk2Car dataset demonstrate the effectiveness of the
proposed methods. Our work offers a deeper insight into the LiDAR-based
grounding task and we expect it presents a promising direction for the
autonomous driving community.