Multi-Agent Deep Reinforcement Learning for Distributed Satellite
  Routing
Federico Lozano-Cuadra Beatriz Soret
This paper introduces a Multi-Agent Deep Reinforcement Learning (MA-DRL)
approach for routing in Low Earth Orbit Satellite Constellations (LSatCs). Each
satellite is an independent decision-making agent with a partial knowledge of
the environment, and supported by feedback received from the nearby agents.
Building on our previous work that introduced a Q-routing solution, the
contribution of this paper is to extend it to a deep learning framework able to
quickly adapt to the network and traffic changes, and based on two phases: (1)
An offline exploration learning phase that relies on a global Deep Neural
Network (DNN) to learn the optimal paths at each possible position and
congestion level; (2) An online exploitation phase with local, on-board,
pre-trained DNNs. Results show that MA-DRL efficiently learns optimal routes
offline that are then loaded for an efficient distributed routing online.