StreetSurf: Extending Multi-view Implicit Surface Reconstruction to
  Street Views
Jianfei Guo Nianchen Deng Xinyang Li Yeqi Bai Botian Shi Chiyu Wang Chenjing Ding Dongliang Wang Yikang Li
We present a novel multi-view implicit surface reconstruction technique,
termed StreetSurf, that is readily applicable to street view images in
widely-used autonomous driving datasets, such as Waymo-perception sequences,
without necessarily requiring LiDAR data. As neural rendering research expands
rapidly, its integration into street views has started to draw interests.
Existing approaches on street views either mainly focus on novel view synthesis
with little exploration of the scene geometry, or rely heavily on dense LiDAR
data when investigating reconstruction. Neither of them investigates multi-view
implicit surface reconstruction, especially under settings without LiDAR data.
Our method extends prior object-centric neural surface reconstruction
techniques to address the unique challenges posed by the unbounded street views
that are captured with non-object-centric, long and narrow camera trajectories.
We delimit the unbounded space into three parts, close-range, distant-view and
sky, with aligned cuboid boundaries, and adapt cuboid/hyper-cuboid hash-grids
along with road-surface initialization scheme for finer and disentangled
representation. To further address the geometric errors arising from
textureless regions and insufficient viewing angles, we adopt geometric priors
that are estimated using general purpose monocular models. Coupled with our
implementation of efficient and fine-grained multi-stage ray marching strategy,
we achieve state of the art reconstruction quality in both geometry and
appearance within only one to two hours of training time with a single RTX3090
GPU for each street view sequence. Furthermore, we demonstrate that the
reconstructed implicit surfaces have rich potential for various downstream
tasks, including ray tracing and LiDAR simulation.