ViHOPE: Visuotactile In-Hand Object 6D Pose Estimation with Shape
  Completion
Hongyu Li Snehal Dikhale Soshi Iba Nawid Jamali
In this letter, we introduce ViHOPE, a novel framework for estimating the 6D
pose of an in-hand object using visuotactile perception. Our key insight is
that the accuracy of the 6D object pose estimate can be improved by explicitly
completing the shape of the object. To this end, we introduce a novel
visuotactile shape completion module that uses a conditional Generative
Adversarial Network to complete the shape of an in-hand object based on
volumetric representation. This approach improves over prior works that
directly regress visuotactile observations to a 6D pose. By explicitly
completing the shape of the in-hand object and jointly optimizing the shape
completion and pose estimation tasks, we improve the accuracy of the 6D object
pose estimate. We train and test our model on a synthetic dataset and compare
it with the state-of-the-art. In the visuotactile shape completion task, we
outperform the state-of-the-art by 265% using the Intersection of Union metric
and achieve 88% lower Chamfer Distance. In the visuotactile pose estimation
task, we present results that suggest our framework reduces position and
angular errors by 35% and 64%, respectively. Furthermore, we ablate our
framework to confirm the gain on the 6D object pose estimate from explicitly
completing the shape. Ultimately, we show that our framework produces models
that are robust to sim-to-real transfer on a real-world robot platform.