Dynamics Generalisation in Reinforcement Learning via Adaptive
  Context-Aware Policies
Michael Beukman Devon Jarvis Richard Klein Steven James Benjamin Rosman
While reinforcement learning has achieved remarkable successes in several
domains, its real-world application is limited due to many methods failing to
generalise to unfamiliar conditions. In this work, we consider the problem of
generalising to new transition dynamics, corresponding to cases in which the
environment's response to the agent's actions differs. For example, the
gravitational force exerted on a robot depends on its mass and changes the
robot's mobility. Consequently, in such cases, it is necessary to condition an
agent's actions on extrinsic state information and pertinent contextual
information reflecting how the environment responds. While the need for
context-sensitive policies has been established, the manner in which context is
incorporated architecturally has received less attention. Thus, in this work,
we present an investigation into how context information should be incorporated
into behaviour learning to improve generalisation. To this end, we introduce a
neural network architecture, the Decision Adapter, which generates the weights
of an adapter module and conditions the behaviour of an agent on the context
information. We show that the Decision Adapter is a useful generalisation of a
previously proposed architecture and empirically demonstrate that it results in
superior generalisation performance compared to previous approaches in several
environments. Beyond this, the Decision Adapter is more robust to irrelevant
distractor variables than several alternative methods.