Federated Reinforcement Learning for Resource Allocation in V2X Networks
Kaidi Xu Shenglong Zhou Geoffrey Ye Li
Resource allocation significantly impacts the performance of
vehicle-to-everything (V2X) networks. Most existing algorithms for resource
allocation are based on optimization or machine learning (e.g., reinforcement
learning). In this paper, we explore resource allocation in a V2X network under
the framework of federated reinforcement learning (FRL). On one hand, the usage
of RL overcomes many challenges from the model-based optimization schemes. On
the other hand, federated learning (FL) enables agents to deal with a number of
practical issues, such as privacy, communication overhead, and exploration
efficiency. The framework of FRL is then implemented by the inexact alternative
direction method of multipliers (ADMM), where subproblems are solved
approximately using policy gradients and accelerated by an adaptive step size
calculated from their second moments. The developed algorithm, PASM, is proven
to be convergent under mild conditions and has a nice numerical performance
compared with some baseline methods for solving the resource allocation problem
in a V2X network.