Scan2LoD3: Reconstructing semantic 3D building models at LoD3 using ray
  casting and Bayesian networks
Olaf Wysocki Yan Xia Magdalena Wysocki Eleonora Grilli Ludwig Hoegner Daniel Cremers Uwe Stilla
Reconstructing semantic 3D building models at the level of detail (LoD) 3 is
a long-standing challenge. Unlike mesh-based models, they require watertight
geometry and object-wise semantics at the fa\c{c}ade level. The principal
challenge of such demanding semantic 3D reconstruction is reliable
fa\c{c}ade-level semantic segmentation of 3D input data. We present a novel
method, called Scan2LoD3, that accurately reconstructs semantic LoD3 building
models by improving fa\c{c}ade-level semantic 3D segmentation. To this end, we
leverage laser physics and 3D building model priors to probabilistically
identify model conflicts. These probabilistic physical conflicts propose
locations of model openings: Their final semantics and shapes are inferred in a
Bayesian network fusing multimodal probabilistic maps of conflicts, 3D point
clouds, and 2D images. To fulfill demanding LoD3 requirements, we use the
estimated shapes to cut openings in 3D building priors and fit semantic 3D
objects from a library of fa\c{c}ade objects. Extensive experiments on the TUM
city campus datasets demonstrate the superior performance of the proposed
Scan2LoD3 over the state-of-the-art methods in fa\c{c}ade-level detection,
semantic segmentation, and LoD3 building model reconstruction. We believe our
method can foster the development of probability-driven semantic 3D
reconstruction at LoD3 since not only the high-definition reconstruction but
also reconstruction confidence becomes pivotal for various applications such as
autonomous driving and urban simulations.