Visuo-Tactile Pretraining for Cable Plugging
Abraham George Selam Gano Pranav Katragadda Amir Barati Farimani
Tactile information is a critical tool for fine-grain manipulation. As
humans, we rely heavily on tactile information to understand objects in our
environments and how to interact with them. We use touch not only to perform
manipulation tasks but also to learn how to perform these tasks. Therefore, to
create robotic agents that can learn to complete manipulation tasks at a human
or super-human level of performance, we need to properly incorporate tactile
information into both skill execution and skill learning. In this paper, we
investigate how we can incorporate tactile information into imitation learning
platforms to improve performance on complex tasks. To do this, we tackle the
challenge of plugging in a USB cable, a dexterous manipulation task that relies
on fine-grain visuo-tactile serving. By incorporating tactile information into
imitation learning frameworks, we are able to train a robotic agent to plug in
a USB cable - a first for imitation learning. Additionally, we explore how
tactile information can be used to train non-tactile agents through a
contrastive-loss pretraining process. Our results show that by pretraining with
tactile information, the performance of a non-tactile agent can be
significantly improved, reaching a level on par with visuo-tactile agents.
  For demonstration videos and access to our codebase, see the project website:
https://sites.google.com/andrew.cmu.edu/visuo-tactile-cable-plugging/home