Scalability of Message Encoding Techniques for Continuous Communication
  Learned with Multi-Agent Reinforcement Learning
Astrid Vanneste Thomas Somers Simon Vanneste Kevin Mets Tom De Schepper Siegfried Mercelis Peter Hellinckx
Many multi-agent systems require inter-agent communication to properly
achieve their goal. By learning the communication protocol alongside the action
protocol using multi-agent reinforcement learning techniques, the agents gain
the flexibility to determine which information should be shared. However, when
the number of agents increases we need to create an encoding of the information
contained in these messages. In this paper, we investigate the effect of
increasing the amount of information that should be contained in a message and
increasing the number of agents. We evaluate these effects on two different
message encoding methods, the mean message encoder and the attention message
encoder. We perform our experiments on a matrix environment. Surprisingly, our
results show that the mean message encoder consistently outperforms the
attention message encoder. Therefore, we analyse the communication protocol
used by the agents that use the mean message encoder and can conclude that the
agents use a combination of an exponential and a logarithmic function in their
communication policy to avoid the loss of important information after applying
the mean message encoder.