Scaling Diffusion Models to Real-World 3D LiDAR Scene Completion
Lucas Nunes Rodrigo Marcuzzi Benedikt Mersch Jens Behley Cyrill Stachniss
Computer vision techniques play a central role in the perception stack of
autonomous vehicles. Such methods are employed to perceive the vehicle
surroundings given sensor data. 3D LiDAR sensors are commonly used to collect
sparse 3D point clouds from the scene. However, compared to human perception,
such systems struggle to deduce the unseen parts of the scene given those
sparse point clouds. In this matter, the scene completion task aims at
predicting the gaps in the LiDAR measurements to achieve a more complete scene
representation. Given the promising results of recent diffusion models as
generative models for images, we propose extending them to achieve scene
completion from a single 3D LiDAR scan. Previous works used diffusion models
over range images extracted from LiDAR data, directly applying image-based
diffusion methods. Distinctly, we propose to directly operate on the points,
reformulating the noising and denoising diffusion process such that it can
efficiently work at scene scale. Together with our approach, we propose a
regularization loss to stabilize the noise predicted during the denoising
process. Our experimental evaluation shows that our method can complete the
scene given a single LiDAR scan as input, producing a scene with more details
compared to state-of-the-art scene completion methods. We believe that our
proposed diffusion process formulation can support further research in
diffusion models applied to scene-scale point cloud data.