Self-Sustaining Multiple Access with Continual Deep Reinforcement
  Learning for Dynamic Metaverse Applications
Hamidreza Mazandarani Masoud Shokrnezhad Tarik Taleb Richard Li
The Metaverse is a new paradigm that aims to create a virtual environment
consisting of numerous worlds, each of which will offer a different set of
services. To deal with such a dynamic and complex scenario, considering the
stringent quality of service requirements aimed at the 6th generation of
communication systems (6G), one potential approach is to adopt self-sustaining
strategies, which can be realized by employing Adaptive Artificial Intelligence
(Adaptive AI) where models are continually re-trained with new data and
conditions. One aspect of self-sustainability is the management of multiple
access to the frequency spectrum. Although several innovative methods have been
proposed to address this challenge, mostly using Deep Reinforcement Learning
(DRL), the problem of adapting agents to a non-stationary environment has not
yet been precisely addressed. This paper fills in the gap in the current
literature by investigating the problem of multiple access in multi-channel
environments to maximize the throughput of the intelligent agent when the
number of active User Equipments (UEs) may fluctuate over time. To solve the
problem, a Double Deep Q-Learning (DDQL) technique empowered by Continual
Learning (CL) is proposed to overcome the non-stationary situation, while the
environment is unknown. Numerical simulations demonstrate that, compared to
other well-known methods, the CL-DDQL algorithm achieves significantly higher
throughputs with a considerably shorter convergence time in highly dynamic
scenarios.