Dexterous Functional Grasping
Ananye Agarwal Shagun Uppal Kenneth Shaw Deepak Pathak
While there have been significant strides in dexterous manipulation, most of
it is limited to benchmark tasks like in-hand reorientation which are of
limited utility in the real world. The main benefit of dexterous hands over
two-fingered ones is their ability to pickup tools and other objects (including
thin ones) and grasp them firmly to apply force. However, this task requires
both a complex understanding of functional affordances as well as precise
low-level control. While prior work obtains affordances from human data this
approach doesn't scale to low-level control. Similarly, simulation training
cannot give the robot an understanding of real-world semantics. In this paper,
we aim to combine the best of both worlds to accomplish functional grasping for
in-the-wild objects. We use a modular approach. First, affordances are obtained
by matching corresponding regions of different objects and then a low-level
policy trained in sim is run to grasp it. We propose a novel application of
eigengrasps to reduce the search space of RL using a small amount of human data
and find that it leads to more stable and physically realistic motion. We find
that eigengrasp action space beats baselines in simulation and outperforms
hardcoded grasping in real and matches or outperforms a trained human
teleoperator. Results visualizations and videos at https://dexfunc.github.io/