CycLight: learning traffic signal cooperation with a cycle-level
  strategy
Gengyue Han Xiaohan Liu Xianyue Peng Hao Wang Yu Han
This study introduces CycLight, a novel cycle-level deep reinforcement
learning (RL) approach for network-level adaptive traffic signal control
(NATSC) systems. Unlike most traditional RL-based traffic controllers that
focus on step-by-step decision making, CycLight adopts a cycle-level strategy,
optimizing cycle length and splits simultaneously using Parameterized Deep
Q-Networks (PDQN) algorithm. This cycle-level approach effectively reduces the
computational burden associated with frequent data communication, meanwhile
enhancing the practicality and safety of real-world applications. A
decentralized framework is formulated for multi-agent cooperation, while
attention mechanism is integrated to accurately assess the impact of the
surroundings on the current intersection. CycLight is tested in a large
synthetic traffic grid using the microscopic traffic simulation tool, SUMO.
Experimental results not only demonstrate the superiority of CycLight over
other state-of-the-art approaches but also showcase its robustness against
information transmission delays.