Select-and-Combine (SAC): A Novel Multi-Stereo Depth Fusion Algorithm
  for Point Cloud Generation via Efficient Local Markov Netlets
Mostafa Elhashash Rongjun Qin
Many practical systems for image-based surface reconstruction employ a
stereo/multi-stereo paradigm, due to its ability to scale for large scenes and
its ease of implementation for out-of-core operations. In this process,
multiple and abundant depth maps from stereo matching must be combined and
fused into a single, consistent, and clean point cloud. However, the noises and
outliers caused by stereo matching and the heterogenous geometric errors of the
poses present a challenge for existing fusion algorithms, since they mostly
assume Gaussian errors and predict fused results based on data from local
spatial neighborhoods, which may inherit uncertainties from multiple depths
resulting in lowered accuracy. In this paper, we propose a novel depth fusion
paradigm, that instead of numerically fusing points from multiple depth maps,
selects the best depth map per point, and combines them into a single and clean
point cloud. This paradigm, called select-and-combine (SAC), is achieved
through modeling the point level fusion using local Markov Netlets, a
micro-network over point across neighboring views for depth/view selection,
followed by a Netlets collapse process for point combination. The Markov
Netlets are optimized such that they can inherently leverage spatial
consistencies among depth maps of neighboring views, thus they can address
errors beyond Gaussian ones. Our experiment results show that our approach
outperforms existing depth fusion approaches by increasing the F1 score that
considers both accuracy and completeness by 2.07% compared to the best existing
method. Finally, our approach generates clearer point clouds that are 18% less
redundant while with a higher accuracy before fusion