DITTO: Demonstration Imitation by Trajectory Transformation
Nick Heppert Max Argus Tim Welschehold Thomas Brox Abhinav Valada
Teaching robots new skills quickly and conveniently is crucial for the
broader adoption of robotic systems. In this work, we address the problem of
one-shot imitation from a single human demonstration, given by an RGB-D video
recording through a two-stage process. In the first stage which is offline, we
extract the trajectory of the demonstration. This entails segmenting
manipulated objects and determining their relative motion in relation to
secondary objects such as containers. Subsequently, in the live online
trajectory generation stage, we first \mbox{re-detect} all objects, then we
warp the demonstration trajectory to the current scene, and finally, we trace
the trajectory with the robot. To complete these steps, our method makes
leverages several ancillary models, including those for segmentation, relative
object pose estimation, and grasp prediction. We systematically evaluate
different combinations of correspondence and re-detection methods to validate
our design decision across a diverse range of tasks. Specifically, we collect
demonstrations of ten different tasks including pick-and-place tasks as well as
articulated object manipulation. Finally, we perform extensive evaluations on a
real robot system to demonstrate the effectiveness and utility of our approach
in real-world scenarios. We make the code publicly available at
http://ditto.cs.uni-freiburg.de.