Double Deep Q-Learning-based Path Selection and Service Placement for
  Latency-Sensitive Beyond 5G Applications
Masoud Shokrnezhad Tarik Taleb Patrizio Dazzi
Nowadays, as the need for capacity continues to grow, entirely novel services
are emerging. A solid cloud-network integrated infrastructure is necessary to
supply these services in a real-time responsive, and scalable way. Due to their
diverse characteristics and limited capacity, communication and computing
resources must be collaboratively managed to unleash their full potential.
Although several innovative methods have been proposed to orchestrate the
resources, most ignored network resources or relaxed the network as a simple
graph, focusing only on cloud resources. This paper fills the gap by studying
the joint problem of communication and computing resource allocation, dubbed
CCRA, including function placement and assignment, traffic prioritization, and
path selection considering capacity constraints and quality requirements, to
minimize total cost. We formulate the problem as a non-linear programming model
and propose two approaches, dubbed B\&B-CCRA and WF-CCRA, based on the Branch
\& Bound and Water-Filling algorithms to solve it when the system is fully
known. Then, for partially known systems, a Double Deep Q-Learning (DDQL)
architecture is designed. Numerical simulations show that B\&B-CCRA optimally
solves the problem, whereas WF-CCRA delivers near-optimal solutions in a
substantially shorter time. Furthermore, it is demonstrated that DDQL-CCRA
obtains near-optimal solutions in the absence of request-specific information.