Continual Robot Learning using Self-Supervised Task Inference
Muhammad Burhan Hafez Stefan Wermter
Endowing robots with the human ability to learn a growing set of skills over
the course of a lifetime as opposed to mastering single tasks is an open
problem in robot learning. While multi-task learning approaches have been
proposed to address this problem, they pay little attention to task inference.
In order to continually learn new tasks, the robot first needs to infer the
task at hand without requiring predefined task representations. In this paper,
we propose a self-supervised task inference approach. Our approach learns
action and intention embeddings from self-organization of the observed movement
and effect parts of unlabeled demonstrations and a higher-level behavior
embedding from self-organization of the joint action-intention embeddings. We
construct a behavior-matching self-supervised learning objective to train a
novel Task Inference Network (TINet) to map an unlabeled demonstration to its
nearest behavior embedding, which we use as the task representation. A
multi-task policy is built on top of the TINet and trained with reinforcement
learning to optimize performance over tasks. We evaluate our approach in the
fixed-set and continual multi-task learning settings with a humanoid robot and
compare it to different multi-task learning baselines. The results show that
our approach outperforms the other baselines, with the difference being more
pronounced in the challenging continual learning setting, and can infer tasks
from incomplete demonstrations. Our approach is also shown to generalize to
unseen tasks based on a single demonstration in one-shot task generalization
experiments.