Rethinking Integration of Prediction and Planning in Deep Learning-Based
  Automated Driving Systems: A Review
Steffen Hagedorn Marcel Hallgarten Martin Stoll Alexandru Condurache
Automated driving has the potential to revolutionize personal, public, and
freight mobility. Besides the enormous challenge of perception, i.e. accurately
perceiving the environment using available sensor data, automated driving
comprises planning a safe, comfortable, and efficient motion trajectory. To
promote safety and progress, many works rely on modules that predict the future
motion of surrounding traffic. Modular automated driving systems commonly
handle prediction and planning as sequential separate tasks. While this
accounts for the influence of surrounding traffic on the ego-vehicle, it fails
to anticipate the reactions of traffic participants to the ego-vehicle's
behavior. Recent works suggest that integrating prediction and planning in an
interdependent joint step is necessary to achieve safe, efficient, and
comfortable driving. While various models implement such integrated systems, a
comprehensive overview and theoretical understanding of different principles
are lacking. We systematically review state-of-the-art deep learning-based
prediction, planning, and integrated prediction and planning models. Different
facets of the integration ranging from model architecture and model design to
behavioral aspects are considered and related to each other. Moreover, we
discuss the implications, strengths, and limitations of different integration
methods. By pointing out research gaps, describing relevant future challenges,
and highlighting trends in the research field, we identify promising directions
for future research.