Energy Efficiency Optimization for Subterranean LoRaWAN Using A
  Reinforcement Learning Approach: A Direct-to-Satellite Scenario
Kaiqiang Lin Muhammad Asad Ullah Hirley Alves Konstantin Mikhaylov Tong Hao
The integration of subterranean LoRaWAN and non-terrestrial networks (NTN)
delivers substantial economic and societal benefits in remote agriculture and
disaster rescue operations. The LoRa modulation leverages quasi-orthogonal
spreading factors (SFs) to optimize data rates, airtime, coverage and energy
consumption. However, it is still challenging to effectively assign SFs to end
devices for minimizing co-SF interference in massive subterranean LoRaWAN NTN.
To address this, we investigate a reinforcement learning (RL)-based SFs
allocation scheme to optimize the system's energy efficiency (EE). To
efficiently capture the device-to-environment interactions in dense networks,
we proposed an SFs allocation technique using the multi-agent dueling double
deep Q-network (MAD3QN) and the multi-agent advantage actor-critic (MAA2C)
algorithms based on an analytical reward mechanism. Our proposed RL-based SFs
allocation approach evinces better performance compared to four benchmarks in
the extreme underground direct-to-satellite scenario. Remarkably, MAD3QN shows
promising potentials in surpassing MAA2C in terms of convergence rate and EE.