Risk-anticipatory autonomous driving strategies considering vehicles'
  weights, based on hierarchical deep reinforcement learning
Di Chen Hao Li Zhicheng Jin Huizhao Tu
Autonomous vehicles (AVs) have the potential to prevent accidents caused by
drivers' error and reduce road traffic risks. Due to the nature of heavy
vehicles, whose collisions cause more serious crashes, the weights of vehicles
need to be considered when making driving strategies aimed at reducing the
potential risks and their consequences in the context of autonomous driving.
This study develops an autonomous driving strategy based on risk anticipation,
considering the weights of surrounding vehicles and using hierarchical deep
reinforcement learning. A risk indicator integrating surrounding vehicles'
weights, based on the risk field theory, is proposed and incorporated into
autonomous driving decisions. A hybrid action space is designed to allow for
left lane changes, right lane changes and car-following, which enables AVs to
act more freely and realistically whenever possible. To solve the above hybrid
decision-making problem, a hierarchical proximal policy optimization (HPPO)
algorithm is developed and an attention mechanism is incorporated, providing
great advantages in maintaining stable performance. An indicator, potential
collision energy in conflicts (PCEC), is newly proposed to evaluate the
performance of the developed AV driving strategy from both the perspectives of
the likelihood and the consequences of potential accidents. An application is
carried out and the simulation results demonstrate that our model provides
driving strategies that reduce both the likelihood and consequences of
potential accidents, at the same time maintaining driving efficiency. The
developed method is especially meaningful for AVs driving on highways, where
heavy vehicles make up a high proportion of the traffic.