Safe Hybrid-Action Reinforcement Learning-Based Decision and Control for
  Discretionary Lane Change
Ruichen Xu Xiao Liu Jinming Xu Yuan Lin
Autonomous lane-change, a key feature of advanced driver-assistance systems,
can enhance traffic efficiency and reduce the incidence of accidents. However,
safe driving of autonomous vehicles remains challenging in complex
environments. How to perform safe and appropriate lane change is a popular
topic of research in the field of autonomous driving. Currently, few papers
consider the safety of reinforcement learning in autonomous lane-change
scenarios. We introduce safe hybrid-action reinforcement learning into
discretionary lane change for the first time and propose Parameterized Soft
Actor-Critic with PID Lagrangian (PASAC-PIDLag) algorithm. Furthermore, we
conduct a comparative analysis of the Parameterized Soft Actor-Critic (PASAC),
which is an unsafe version of PASAC-PIDLag. Both algorithms are employed to
train the lane-change strategy of autonomous vehicles to output discrete
lane-change decision and longitudinal vehicle acceleration. Our simulation
results indicate that at a traffic density of 15 vehicles per kilometer (15
veh/km), the PASAC-PIDLag algorithm exhibits superior safety with a collision
rate of 0%, outperforming the PASAC algorithm, which has a collision rate of
1%. The outcomes of the generalization assessments reveal that at low traffic
density levels, both the PASAC-PIDLag and PASAC algorithms are proficient in
attaining a 0% collision rate. Under conditions of high traffic flow density,
the PASAC-PIDLag algorithm surpasses PASAC in terms of both safety and
optimality.