Learning to Cooperate and Communicate Over Imperfect Channels
Jannis Weil Gizem Ekinci Heinz Koeppl Tobias Meuser
Information exchange in multi-agent systems improves the cooperation among
agents, especially in partially observable settings. In the real world,
communication is often carried out over imperfect channels. This requires
agents to handle uncertainty due to potential information loss. In this paper,
we consider a cooperative multi-agent system where the agents act and exchange
information in a decentralized manner using a limited and unreliable channel.
To cope with such channel constraints, we propose a novel communication
approach based on independent Q-learning. Our method allows agents to
dynamically adapt how much information to share by sending messages of
different sizes, depending on their local observations and the channel's
properties. In addition to this message size selection, agents learn to encode
and decode messages to improve their jointly trained policies. We show that our
approach outperforms approaches without adaptive capabilities in a novel
cooperative digit-prediction environment and discuss its limitations in the
traffic junction environment.