Learning Reusable Manipulation Strategies
Jiayuan Mao Joshua B. Tenenbaum Tomás Lozano-Pérez Leslie Pack Kaelbling
Humans demonstrate an impressive ability to acquire and generalize
manipulation "tricks." Even from a single demonstration, such as using soup
ladles to reach for distant objects, we can apply this skill to new scenarios
involving different object positions, sizes, and categories (e.g., forks and
hammers). Additionally, we can flexibly combine various skills to devise
long-term plans. In this paper, we present a framework that enables machines to
acquire such manipulation skills, referred to as "mechanisms," through a single
demonstration and self-play. Our key insight lies in interpreting each
demonstration as a sequence of changes in robot-object and object-object
contact modes, which provides a scaffold for learning detailed samplers for
continuous parameters. These learned mechanisms and samplers can be seamlessly
integrated into standard task and motion planners, enabling their compositional
use.