Federated Multi-Agent Deep Reinforcement Learning for Dynamic and
  Flexible 3D Operation of 5G Multi-MAP Networks
Esteban Catté Mohamed Sana Mickael Maman
This paper addresses the efficient management of Mobile Access Points (MAPs),
which are Unmanned Aerial Vehicles (UAV), in 5G networks. We propose a
two-level hierarchical architecture, which dynamically reconfigures the network
while considering Integrated Access-Backhaul (IAB) constraints. The high-layer
decision process determines the number of MAPs through consensus, and we
develop a joint optimization process to account for co-dependence in network
self-management. In the low-layer, MAPs manage their placement using a
double-attention based Deep Reinforcement Learning (DRL) model that encourages
cooperation without retraining. To improve generalization and reduce
complexity, we propose a federated mechanism for training and sharing one
placement model for every MAP in the low-layer. Additionally, we jointly
optimize the placement and backhaul connectivity of MAPs using a
multi-objective reward function, considering the impact of varying MAP
placement on wireless backhaul connectivity.