Bridging the Reality Gap of Reinforcement Learning based Traffic Signal
  Control using Domain Randomization and Meta Learning
Arthur Müller Matthia Sabatelli
Reinforcement Learning (RL) has been widely explored in Traffic Signal
Control (TSC) applications, however, still no such system has been deployed in
practice. A key barrier to progress in this area is the reality gap, the
discrepancy that results from differences between simulation models and their
real-world equivalents. In this paper, we address this challenge by first
presenting a comprehensive analysis of potential simulation parameters that
contribute to this reality gap. We then also examine two promising strategies
that can bridge this gap: Domain Randomization (DR) and Model-Agnostic
Meta-Learning (MAML). Both strategies were trained with a traffic simulation
model of an intersection. In addition, the model was embedded in LemgoRL, a
framework that integrates realistic, safety-critical requirements into the
control system. Subsequently, we evaluated the performance of the two methods
on a separate model of the same intersection that was developed with a
different traffic simulator. In this way, we mimic the reality gap. Our
experimental results show that both DR and MAML outperform a state-of-the-art
RL algorithm, therefore highlighting their potential to mitigate the reality
gap in RLbased TSC systems.