StructuredMesh: 3D Structured Optimization of Façade Components on
  Photogrammetric Mesh Models using Binary Integer Programming
Libin Wang Han Hu Qisen Shang Bo Xu Qing Zhu
The lack of fa\c{c}ade structures in photogrammetric mesh models renders them
inadequate for meeting the demands of intricate applications. Moreover, these
mesh models exhibit irregular surfaces with considerable geometric noise and
texture quality imperfections, making the restoration of structures
challenging. To address these shortcomings, we present StructuredMesh, a novel
approach for reconstructing fa\c{c}ade structures conforming to the regularity
of buildings within photogrammetric mesh models. Our method involves capturing
multi-view color and depth images of the building model using a virtual camera
and employing a deep learning object detection pipeline to semi-automatically
extract the bounding boxes of fa\c{c}ade components such as windows, doors, and
balconies from the color image. We then utilize the depth image to remap these
boxes into 3D space, generating an initial fa\c{c}ade layout. Leveraging
architectural knowledge, we apply binary integer programming (BIP) to optimize
the 3D layout's structure, encompassing the positions, orientations, and sizes
of all components. The refined layout subsequently informs fa\c{c}ade modeling
through instance replacement. We conducted experiments utilizing building mesh
models from three distinct datasets, demonstrating the adaptability,
robustness, and noise resistance of our proposed methodology. Furthermore, our
3D layout evaluation metrics reveal that the optimized layout enhances
precision, recall, and F-score by 6.5%, 4.5%, and 5.5%, respectively, in
comparison to the initial layout.