ZS6D: Zero-shot 6D Object Pose Estimation using Vision Transformers
Philipp Ausserlechner David Haberger Stefan Thalhammer Jean-Baptiste Weibel Markus Vincze
As robotic systems increasingly encounter complex and unconstrained
real-world scenarios, there is a demand to recognize diverse objects. The
state-of-the-art 6D object pose estimation methods rely on object-specific
training and therefore do not generalize to unseen objects. Recent novel object
pose estimation methods are solving this issue using task-specific fine-tuned
CNNs for deep template matching. This adaptation for pose estimation still
requires expensive data rendering and training procedures. MegaPose for example
is trained on a dataset consisting of two million images showing 20,000
different objects to reach such generalization capabilities. To overcome this
shortcoming we introduce ZS6D, for zero-shot novel object 6D pose estimation.
Visual descriptors, extracted using pre-trained Vision Transformers (ViT), are
used for matching rendered templates against query images of objects and for
establishing local correspondences. These local correspondences enable deriving
geometric correspondences and are used for estimating the object's 6D pose with
RANSAC-based PnP. This approach showcases that the image descriptors extracted
by pre-trained ViTs are well-suited to achieve a notable improvement over two
state-of-the-art novel object 6D pose estimation methods, without the need for
task-specific fine-tuning. Experiments are performed on LMO, YCBV, and TLESS.
In comparison to one of the two methods we improve the Average Recall on all
three datasets and compared to the second method we improve on two datasets.