UniWorld: Autonomous Driving Pre-training via World Models
Chen Min Dawei Zhao Liang Xiao Yiming Nie Bin Dai
In this paper, we draw inspiration from Alberto Elfes' pioneering work in
1989, where he introduced the concept of the occupancy grid as World Models for
robots. We imbue the robot with a spatial-temporal world model, termed
UniWorld, to perceive its surroundings and predict the future behavior of other
participants. UniWorld involves initially predicting 4D geometric occupancy as
the World Models for foundational stage and subsequently fine-tuning on
downstream tasks. UniWorld can estimate missing information concerning the
world state and predict plausible future states of the world. Besides,
UniWorld's pre-training process is label-free, enabling the utilization of
massive amounts of image-LiDAR pairs to build a Foundational Model.The proposed
unified pre-training framework demonstrates promising results in key tasks such
as motion prediction, multi-camera 3D object detection, and surrounding
semantic scene completion. When compared to monocular pre-training methods on
the nuScenes dataset, UniWorld shows a significant improvement of about 1.5% in
IoU for motion prediction, 2.0% in mAP and 2.0% in NDS for multi-camera 3D
object detection, as well as a 3% increase in mIoU for surrounding semantic
scene completion. By adopting our unified pre-training method, a 25% reduction
in 3D training annotation costs can be achieved, offering significant practical
value for the implementation of real-world autonomous driving. Codes are
publicly available at https://github.com/chaytonmin/UniWorld.