Learning Decentralized Traffic Signal Controllers with Multi-Agent Graph
  Reinforcement Learning
Yao Zhang Zhiwen Yu Jun Zhang Liang Wang Tom H. Luan Bin Guo Chau Yuen
This paper considers optimal traffic signal control in smart cities, which
has been taken as a complex networked system control problem. Given the
interacting dynamics among traffic lights and road networks, attaining
controller adaptivity and scalability stands out as a primary challenge.
Capturing the spatial-temporal correlation among traffic lights under the
framework of Multi-Agent Reinforcement Learning (MARL) is a promising solution.
Nevertheless, existing MARL algorithms ignore effective information aggregation
which is fundamental for improving the learning capacity of decentralized
agents. In this paper, we design a new decentralized control architecture with
improved environmental observability to capture the spatial-temporal
correlation. Specifically, we first develop a topology-aware information
aggregation strategy to extract correlation-related information from
unstructured data gathered in the road network. Particularly, we transfer the
road network topology into a graph shift operator by forming a diffusion
process on the topology, which subsequently facilitates the construction of
graph signals. A diffusion convolution module is developed, forming a new MARL
algorithm, which endows agents with the capabilities of graph learning.
Extensive experiments based on both synthetic and real-world datasets verify
that our proposal outperforms existing decentralized algorithms.