Distributed Autonomous Swarm Formation for Dynamic Network Bridging
Raffaele Galliera Thies Möhlenhof Alessandro Amato Daniel Duran Kristen Brent Venable Niranjan Suri
Effective operation and seamless cooperation of robotic systems are a
fundamental component of next-generation technologies and applications. In
contexts such as disaster response, swarm operations require coordinated
behavior and mobility control to be handled in a distributed manner, with the
quality of the agents' actions heavily relying on the communication between
them and the underlying network. In this paper, we formulate the problem of
dynamic network bridging in a novel Decentralized Partially Observable Markov
Decision Process (Dec-POMDP), where a swarm of agents cooperates to form a link
between two distant moving targets. Furthermore, we propose a Multi-Agent
Reinforcement Learning (MARL) approach for the problem based on Graph
Convolutional Reinforcement Learning (DGN) which naturally applies to the
networked, distributed nature of the task. The proposed method is evaluated in
a simulated environment and compared to a centralized heuristic baseline
showing promising results. Moreover, a further step in the direction of
sim-to-real transfer is presented, by additionally evaluating the proposed
approach in a near Live Virtual Constructive (LVC) UAV framework.