Physics-Based Trajectory Design for Cellular-Connected UAV in Rainy
  Environments Based on Deep Reinforcement Learning
Hao Qin Zhaozhou Wu Xingqi Zhang
Cellular-connected unmanned aerial vehicles (UAVs) have gained increasing
attention due to their potential to enhance conventional UAV capabilities by
leveraging existing cellular infrastructure for reliable communications between
UAVs and base stations. They have been used for various applications, including
weather forecasting and search and rescue operations. However, under extreme
weather conditions such as rainfall, it is challenging for the trajectory
design of cellular UAVs, due to weak coverage regions in the sky, limitations
of UAV flying time, and signal attenuation caused by raindrops. To this end,
this paper proposes a physics-based trajectory design approach for
cellular-connected UAVs in rainy environments. A physics-based electromagnetic
simulator is utilized to take into account detailed environment information and
the impact of rain on radio wave propagation. The trajectory optimization
problem is formulated to jointly consider UAV flying time and
signal-to-interference ratio, and is solved through a Markov decision process
using deep reinforcement learning algorithms based on multi-step learning and
double Q-learning. Optimal UAV trajectories are compared in examples with
homogeneous atmosphere medium and rain medium. Additionally, a thorough study
of varying weather conditions on trajectory design is provided, and the impact
of weight coefficients in the problem formulation is discussed. The proposed
approach has demonstrated great potential for UAV trajectory design under rainy
weather conditions.