PseudoTouch: Efficiently Imaging the Surface Feel of Objects for Robotic
  Manipulation
Adrian Röfer Nick Heppert Abdallah Ayman Eugenio Chisari Abhinav Valada
Humans seemingly incorporate potential touch signals in their perception. Our
goal is to equip robots with a similar capability, which we term \ourmodel.
\ourmodel aims to predict the expected touch signal based on a visual patch
representing the touched area. We frame this problem as the task of learning a
low-dimensional visual-tactile embedding, wherein we encode a depth patch from
which we decode the tactile signal. To accomplish this task, we employ ReSkin,
an inexpensive and replaceable magnetic-based tactile sensor. Using ReSkin, we
collect and train PseudoTouch on a dataset comprising aligned tactile and
visual data pairs obtained through random touching of eight basic geometric
shapes. We demonstrate the efficacy of PseudoTouch through its application to
two downstream tasks: object recognition and grasp stability prediction. In the
object recognition task, we evaluate the learned embedding's performance on a
set of five basic geometric shapes and five household objects. Using
PseudoTouch, we achieve an object recognition accuracy 84% after just ten
touches, surpassing a proprioception baseline. For the grasp stability task, we
use ACRONYM labels to train and evaluate a grasp success predictor using
PseudoTouch's predictions derived from virtual depth information. Our approach
yields an impressive 32% absolute improvement in accuracy compared to the
baseline relying on partial point cloud data. We make the data, code, and
trained models publicly available at http://pseudotouch.cs.uni-freiburg.de.