Discovering Individual Rewards in Collective Behavior through Inverse
  Multi-Agent Reinforcement Learning
Daniel Waelchli Pascal Weber Petros Koumoutsakos
The discovery of individual objectives in collective behavior of complex
dynamical systems such as fish schools and bacteria colonies is a long-standing
challenge. Inverse reinforcement learning is a potent approach for addressing
this challenge but its applicability to dynamical systems, involving continuous
state-action spaces and multiple interacting agents, has been limited. In this
study, we tackle this challenge by introducing an off-policy inverse
multi-agent reinforcement learning algorithm (IMARL). Our approach combines the
ReF-ER techniques with guided cost learning. By leveraging demonstrations, our
algorithm automatically uncovers the reward function and learns an effective
policy for the agents. Through extensive experimentation, we demonstrate that
the proposed policy captures the behavior observed in the provided data, and
achieves promising results across problem domains including single agent models
in the OpenAI gym and multi-agent models of schooling behavior. The present
study shows that the proposed IMARL algorithm is a significant step towards
understanding collective dynamics from the perspective of its constituents, and
showcases its value as a tool for studying complex physical systems exhibiting
collective behaviour.