Extreme Two-View Geometry From Object Poses with Diffusion Models
Yujing Sun Caiyi Sun Yuan Liu Yuexin Ma Siu Ming Yiu
Human has an incredible ability to effortlessly perceive the viewpoint
difference between two images containing the same object, even when the
viewpoint change is astonishingly vast with no co-visible regions in the
images. This remarkable skill, however, has proven to be a challenge for
existing camera pose estimation methods, which often fail when faced with large
viewpoint differences due to the lack of overlapping local features for
matching. In this paper, we aim to effectively harness the power of object
priors to accurately determine two-view geometry in the face of extreme
viewpoint changes. In our method, we first mathematically transform the
relative camera pose estimation problem to an object pose estimation problem.
Then, to estimate the object pose, we utilize the object priors learned from a
diffusion model Zero123 to synthesize novel-view images of the object. The
novel-view images are matched to determine the object pose and thus the
two-view camera pose. In experiments, our method has demonstrated extraordinary
robustness and resilience to large viewpoint changes, consistently estimating
two-view poses with exceptional generalization ability across both synthetic
and real-world datasets. Code will be available at
https://github.com/scy639/Extreme-Two-View-Geometry-From-Object-Poses-with-Diffusion-Models.