MADRL-based UAVs Trajectory Design with Anti-Collision Mechanism in
  Vehicular Networks
Leonardo Spampinato Enrico Testi Chiara Buratti Riccardo Marini
In upcoming 6G networks, unmanned aerial vehicles (UAVs) are expected to play
a fundamental role by acting as mobile base stations, particularly for
demanding vehicle-to-everything (V2X) applications. In this scenario, one of
the most challenging problems is the design of trajectories for multiple UAVs,
cooperatively serving the same area. Such joint trajectory design can be
performed using multi-agent deep reinforcement learning (MADRL) algorithms, but
ensuring collision-free paths among UAVs becomes a critical challenge.
Traditional methods involve imposing high penalties during training to
discourage unsafe conditions, but these can be proven to be ineffective,
whereas binary masks can be used to restrict unsafe actions, but naively
applying them to all agents can lead to suboptimal solutions and
inefficiencies. To address these issues, we propose a rank-based binary masking
approach. Higher-ranked UAVs move optimally, while lower-ranked UAVs use this
information to define improved binary masks, reducing the number of unsafe
actions. This approach allows to obtain a good trade-off between exploration
and exploitation, resulting in enhanced training performance, while maintaining
safety constraints.