Detector-Free Structure from Motion
Xingyi He Jiaming Sun Yifan Wang Sida Peng Qixing Huang Hujun Bao Xiaowei Zhou
We propose a new structure-from-motion framework to recover accurate camera
poses and point clouds from unordered images. Traditional SfM systems typically
rely on the successful detection of repeatable keypoints across multiple views
as the first step, which is difficult for texture-poor scenes, and poor
keypoint detection may break down the whole SfM system. We propose a new
detector-free SfM framework to draw benefits from the recent success of
detector-free matchers to avoid the early determination of keypoints, while
solving the multi-view inconsistency issue of detector-free matchers.
Specifically, our framework first reconstructs a coarse SfM model from
quantized detector-free matches. Then, it refines the model by a novel
iterative refinement pipeline, which iterates between an attention-based
multi-view matching module to refine feature tracks and a geometry refinement
module to improve the reconstruction accuracy. Experiments demonstrate that the
proposed framework outperforms existing detector-based SfM systems on common
benchmark datasets. We also collect a texture-poor SfM dataset to demonstrate
the capability of our framework to reconstruct texture-poor scenes. Based on
this framework, we take $\textit{first place}$ in Image Matching Challenge
2023.