Few-Shot In-Context Imitation Learning via Implicit Graph Alignment
Vitalis Vosylius Edward Johns
Consider the following problem: given a few demonstrations of a task across a
few different objects, how can a robot learn to perform that same task on new,
previously unseen objects? This is challenging because the large variety of
objects within a class makes it difficult to infer the task-relevant
relationship between the new objects and the objects in the demonstrations. We
address this by formulating imitation learning as a conditional alignment
problem between graph representations of objects. Consequently, we show that
this conditioning allows for in-context learning, where a robot can perform a
task on a set of new objects immediately after the demonstrations, without any
prior knowledge about the object class or any further training. In our
experiments, we explore and validate our design choices, and we show that our
method is highly effective for few-shot learning of several real-world,
everyday tasks, whilst outperforming baselines. Videos are available on our
project webpage at https://www.robot-learning.uk/implicit-graph-alignment.