MV-DeepSDF: Implicit Modeling with Multi-Sweep Point Clouds for 3D
  Vehicle Reconstruction in Autonomous Driving
Yibo Liu Kelly Zhu Guile Wu Yuan Ren Bingbing Liu Yang Liu Jinjun Shan
Reconstructing 3D vehicles from noisy and sparse partial point clouds is of
great significance to autonomous driving. Most existing 3D reconstruction
methods cannot be directly applied to this problem because they are elaborately
designed to deal with dense inputs with trivial noise. In this work, we propose
a novel framework, dubbed MV-DeepSDF, which estimates the optimal Signed
Distance Function (SDF) shape representation from multi-sweep point clouds to
reconstruct vehicles in the wild. Although there have been some SDF-based
implicit modeling methods, they only focus on single-view-based reconstruction,
resulting in low fidelity. In contrast, we first analyze multi-sweep
consistency and complementarity in the latent feature space and propose to
transform the implicit space shape estimation problem into an element-to-set
feature extraction problem. Then, we devise a new architecture to extract
individual element-level representations and aggregate them to generate a
set-level predicted latent code. This set-level latent code is an expression of
the optimal 3D shape in the implicit space, and can be subsequently decoded to
a continuous SDF of the vehicle. In this way, our approach learns consistent
and complementary information among multi-sweeps for 3D vehicle reconstruction.
We conduct thorough experiments on two real-world autonomous driving datasets
(Waymo and KITTI) to demonstrate the superiority of our approach over
state-of-the-art alternative methods both qualitatively and quantitatively.