Distributed Policy Gradient for Linear Quadratic Networked Control with
  Limited Communication Range
Yuzi Yan Yuan Shen
This paper proposes a scalable distributed policy gradient method and proves
its convergence to near-optimal solution in multi-agent linear quadratic
networked systems. The agents engage within a specified network under local
communication constraints, implying that each agent can only exchange
information with a limited number of neighboring agents. On the underlying
graph of the network, each agent implements its control input depending on its
nearby neighbors' states in the linear quadratic control setting. We show that
it is possible to approximate the exact gradient only using local information.
Compared with the centralized optimal controller, the performance gap decreases
to zero exponentially as the communication and control ranges increase. We also
demonstrate how increasing the communication range enhances system stability in
the gradient descent process, thereby elucidating a critical trade-off. The
simulation results verify our theoretical findings.