FoundPose: Unseen Object Pose Estimation with Foundation Features
Evin Pınar Örnek Yann Labbé Bugra Tekin Lingni Ma Cem Keskin Christian Forster Tomas Hodan
We propose FoundPose, a method for 6D pose estimation of unseen rigid objects
from a single RGB image. The method assumes that 3D models of the objects are
available but does not require any object-specific training. This is achieved
by building upon DINOv2, a recent vision foundation model with impressive
generalization capabilities. An online pose estimation stage is supported by a
minimal object representation that is built during a short onboarding stage
from DINOv2 patch features extracted from rendered object templates. Given a
query image with an object segmentation mask, FoundPose first rapidly retrieves
a handful of similarly looking templates by a DINOv2-based bag-of-words
approach. Pose hypotheses are then generated from 2D-3D correspondences
established by matching DINOv2 patch features between the query image and a
retrieved template, and finally optimized by featuremetric refinement. The
method can handle diverse objects, including challenging ones with symmetries
and without any texture, and noticeably outperforms existing RGB methods for
coarse pose estimation in both accuracy and speed on the standard BOP
benchmark. With the featuremetric and additional MegaPose refinement, which are
demonstrated complementary, the method outperforms all RGB competitors. Source
code is at: evinpinar.github.io/foundpose.