Emergence of Chemotactic Strategies with Multi-Agent Reinforcement
  Learning
Samuel Tovey Christoph Lohrmann Christian Holm
Reinforcement learning (RL) is a flexible and efficient method for
programming micro-robots in complex environments. Here we investigate whether
reinforcement learning can provide insights into biological systems when
trained to perform chemotaxis. Namely, whether we can learn about how
intelligent agents process given information in order to swim towards a target.
We run simulations covering a range of agent shapes, sizes, and swim speeds to
determine if the physical constraints on biological swimmers, namely Brownian
motion, lead to regions where reinforcement learners' training fails. We find
that the RL agents can perform chemotaxis as soon as it is physically possible
and, in some cases, even before the active swimming overpowers the stochastic
environment. We study the efficiency of the emergent policy and identify
convergence in agent size and swim speeds. Finally, we study the strategy
adopted by the reinforcement learning algorithm to explain how the agents
perform their tasks. To this end, we identify three emerging dominant
strategies and several rare approaches taken. These strategies, whilst
producing almost identical trajectories in simulation, are distinct and give
insight into the possible mechanisms behind which biological agents explore
their environment and respond to changing conditions.