Collaborative Policy Learning for Dynamic Scheduling Tasks in
  Cloud-Edge-Terminal IoT Networks Using Federated Reinforcement Learning
Do-Yup Kim Da-Eun Lee Ji-Wan Kim Hyun-Suk Lee
In this paper, we examine cloud-edge-terminal IoT networks, where edges
undertake a range of typical dynamic scheduling tasks. In these IoT networks, a
central policy for each task can be constructed at a cloud server. The central
policy can be then used by the edges conducting the task, thereby mitigating
the need for them to learn their own policy from scratch. Furthermore, this
central policy can be collaboratively learned at the cloud server by
aggregating local experiences from the edges, thanks to the hierarchical
architecture of the IoT networks. To this end, we propose a novel collaborative
policy learning framework for dynamic scheduling tasks using federated
reinforcement learning. For effective learning, our framework adaptively
selects the tasks for collaborative learning in each round, taking into account
the need for fairness among tasks. In addition, as a key enabler of the
framework, we propose an edge-agnostic policy structure that enables the
aggregation of local policies from different edges. We then provide the
convergence analysis of the framework. Through simulations, we demonstrate that
our proposed framework significantly outperforms the approaches without
collaborative policy learning. Notably, it accelerates the learning speed of
the policies and allows newly arrived edges to adapt to their tasks more
easily.