LiDAR-BEVMTN: Real-Time LiDAR Bird's-Eye View Multi-Task Perception
  Network for Autonomous Driving
Sambit Mohapatra Senthil Yogamani Varun Ravi Kumar Stefan Milz Heinrich Gotzig Patrick Mäder
LiDAR is crucial for robust 3D scene perception in autonomous driving. LiDAR
perception has the largest body of literature after camera perception. However,
multi-task learning across tasks like detection, segmentation, and motion
estimation using LiDAR remains relatively unexplored, especially on
automotive-grade embedded platforms. We present a real-time multi-task
convolutional neural network for LiDAR-based object detection, semantics, and
motion segmentation. The unified architecture comprises a shared encoder and
task-specific decoders, enabling joint representation learning. We propose a
novel Semantic Weighting and Guidance (SWAG) module to transfer semantic
features for improved object detection selectively. Our heterogeneous training
scheme combines diverse datasets and exploits complementary cues between tasks.
The work provides the first embedded implementation unifying these key
perception tasks from LiDAR point clouds achieving 3ms latency on the embedded
NVIDIA Xavier platform. We achieve state-of-the-art results for two tasks,
semantic and motion segmentation, and close to state-of-the-art performance for
3D object detection. By maximizing hardware efficiency and leveraging
multi-task synergies, our method delivers an accurate and efficient solution
tailored for real-world automated driving deployment. Qualitative results can
be seen at https://youtu.be/H-hWRzv2lIY.