Inter-Cell Network Slicing With Transfer Learning Empowered Multi-Agent
  Deep Reinforcement Learning
Tianlun Hu Qi Liao Qiang Liu Georg Carle
Network slicing enables operators to efficiently support diverse applications
on a common physical infrastructure. The ever-increasing densification of
network deployment leads to complex and non-trivial inter-cell interference,
which requires more than inaccurate analytic models to dynamically optimize
resource management for network slices. In this paper, we develop a DIRP
algorithm with multiple deep reinforcement learning (DRL) agents to
cooperatively optimize resource partition in individual cells to fulfill the
requirements of each slice, based on two alternative reward functions.
Nevertheless, existing DRL approaches usually tie the pretrained model
parameters to specific network environments with poor transferability, which
raises practical deployment concerns in large-scale mobile networks. Hence, we
design a novel transfer learning-aided DIRP (TL-DIRP) algorithm to ease the
transfer of DIRP agents across different network environments in terms of
sample efficiency, model reproducibility, and algorithm scalability. The
TL-DIRP algorithm first centrally trains a generalized model and then transfers
the "generalist" to each local agent as "specialist" with distributed
finetuning and execution. TL-DIRP consists of two steps: 1) centralized
training of a generalized distributed model, 2) transferring the "generalist"
to each "specialist" with distributed finetuning and execution. The numerical
results show that not only DIRP outperforms existing baseline approaches in
terms of faster convergence and higher reward, but more importantly, TL-DIRP
significantly improves the service performance, with reduced exploration cost,
accelerated convergence rate, and enhanced model reproducibility. As compared
to a traffic-aware baseline, TL-DIRP provides about 15% less violation ratio of
the quality of service (QoS) for the worst slice service and 8.8% less
violation on the average service QoS.