Sparse Points to Dense Clouds: Enhancing 3D Detection with Limited LiDAR
  Data
Aakash Kumar Chen Chen Ajmal Mian Neils Lobo Mubarak Shah
3D detection is a critical task that enables machines to identify and locate
objects in three-dimensional space. It has a broad range of applications in
several fields, including autonomous driving, robotics and augmented reality.
Monocular 3D detection is attractive as it requires only a single camera,
however, it lacks the accuracy and robustness required for real world
applications. High resolution LiDAR on the other hand, can be expensive and
lead to interference problems in heavy traffic given their active
transmissions. We propose a balanced approach that combines the advantages of
monocular and point cloud-based 3D detection. Our method requires only a small
number of 3D points, that can be obtained from a low-cost, low-resolution
sensor. Specifically, we use only 512 points, which is just 1% of a full LiDAR
frame in the KITTI dataset. Our method reconstructs a complete 3D point cloud
from this limited 3D information combined with a single image. The
reconstructed 3D point cloud and corresponding image can be used by any
multi-modal off-the-shelf detector for 3D object detection. By using the
proposed network architecture with an off-the-shelf multi-modal 3D detector,
the accuracy of 3D detection improves by 20% compared to the state-of-the-art
monocular detection methods and 6% to 9% compare to the baseline multi-modal
methods on KITTI and JackRabbot datasets.