Perimeter Control Using Deep Reinforcement Learning: A Model-free
  Approach towards Homogeneous Flow Rate Optimization
Xiaocan Li Ray Coden Mercurius Ayal Taitler Xiaoyu Wang Mohammad Noaeen Scott Sanner Baher Abdulhai
Perimeter control maintains high traffic efficiency within protected regions
by controlling transfer flows among regions to ensure that their traffic
densities are below critical values. Existing approaches can be categorized as
either model-based or model-free, depending on whether they rely on network
transmission models (NTMs) and macroscopic fundamental diagrams (MFDs).
Although model-based approaches are more data efficient and have performance
guarantees, they are inherently prone to model bias and inaccuracy. For
example, NTMs often become imprecise for a large number of protected regions,
and MFDs can exhibit scatter and hysteresis that are not captured in existing
model-based works. Moreover, no existing studies have employed reinforcement
learning for homogeneous flow rate optimization in microscopic simulation,
where spatial characteristics, vehicle-level information, and metering
realizations -- often overlooked in macroscopic simulations -- are taken into
account. To circumvent issues of model-based approaches and macroscopic
simulation, we propose a model-free deep reinforcement learning approach that
optimizes the flow rate homogeneously at the perimeter at the microscopic
level. Results demonstrate that our model-free reinforcement learning approach
without any knowledge of NTMs or MFDs can compete and match the performance of
a model-based approach, and exhibits enhanced generalizability and scalability.