Action Inference by Maximising Evidence: Zero-Shot Imitation from
  Observation with World Models
Xingyuan Zhang Philip Becker-Ehmck Patrick van der Smagt Maximilian Karl
Unlike most reinforcement learning agents which require an unrealistic amount
of environment interactions to learn a new behaviour, humans excel at learning
quickly by merely observing and imitating others. This ability highly depends
on the fact that humans have a model of their own embodiment that allows them
to infer the most likely actions that led to the observed behaviour. In this
paper, we propose Action Inference by Maximising Evidence (AIME) to replicate
this behaviour using world models. AIME consists of two distinct phases. In the
first phase, the agent learns a world model from its past experience to
understand its own body by maximising the ELBO. While in the second phase, the
agent is given some observation-only demonstrations of an expert performing a
novel task and tries to imitate the expert's behaviour. AIME achieves this by
defining a policy as an inference model and maximising the evidence of the
demonstration under the policy and world model. Our method is "zero-shot" in
the sense that it does not require further training for the world model or
online interactions with the environment after given the demonstration. We
empirically validate the zero-shot imitation performance of our method on the
Walker and Cheetah embodiment of the DeepMind Control Suite and find it
outperforms the state-of-the-art baselines. Code is available at:
https://github.com/argmax-ai/aime.