SD-MVS: Segmentation-Driven Deformation Multi-View Stereo with Spherical
  Refinement and EM optimization
Zhenlong Yuan Jiakai Cao Zhaoxin Li Hao Jiang Zhaoqi Wang
In this paper, we introduce Segmentation-Driven Deformation Multi-View Stereo
(SD-MVS), a method that can effectively tackle challenges in 3D reconstruction
of textureless areas. We are the first to adopt the Segment Anything Model
(SAM) to distinguish semantic instances in scenes and further leverage these
constraints for pixelwise patch deformation on both matching cost and
propagation. Concurrently, we propose a unique refinement strategy that
combines spherical coordinates and gradient descent on normals and pixelwise
search interval on depths, significantly improving the completeness of
reconstructed 3D model. Furthermore, we adopt the Expectation-Maximization (EM)
algorithm to alternately optimize the aggregate matching cost and
hyperparameters, effectively mitigating the problem of parameters being
excessively dependent on empirical tuning. Evaluations on the ETH3D
high-resolution multi-view stereo benchmark and the Tanks and Temples dataset
demonstrate that our method can achieve state-of-the-art results with less time
consumption.