Joint Service Caching, Communication and Computing Resource Allocation
  in Collaborative MEC Systems: A DRL-based Two-timescale Approach
Qianqian Liu Haixia Zhang Xin Zhang Dongfeng Yuan
Meeting the strict Quality of Service (QoS) requirements of terminals has
imposed a signiffcant challenge on Multiaccess Edge Computing (MEC) systems,
due to the limited multidimensional resources. To address this challenge, we
propose a collaborative MEC framework that facilitates resource sharing between
the edge servers, and with the aim to maximize the long-term QoS and reduce the
cache switching cost through joint optimization of service caching,
collaborative offfoading, and computation and communication resource
allocation. The dual timescale feature and temporal recurrence relationship
between service caching and other resource allocation make solving the problem
even more challenging. To solve it, we propose a deep reinforcement learning
(DRL)-based dual timescale scheme, called DGL-DDPG, which is composed of a
short-term genetic algorithm (GA) and a long short-term memory network-based
deep deterministic policy gradient (LSTM-DDPG). In doing so, we reformulate the
optimization problem as a Markov decision process (MDP) where the
small-timescale resource allocation decisions generated by an improved GA are
taken as the states and input into a centralized LSTM-DDPG agent to generate
the service caching decision for the large-timescale. Simulation results
demonstrate that our proposed algorithm outperforms the baseline algorithms in
terms of the average QoS and cache switching cost.