A Survey for Foundation Models in Autonomous Driving
Haoxiang Gao Yaqian Li Kaiwen Long Ming Yang Yiqing Shen
The advent of foundation models has revolutionized the fields of natural
language processing and computer vision, paving the way for their application
in autonomous driving (AD). This survey presents a comprehensive review of more
than 40 research papers, demonstrating the role of foundation models in
enhancing AD. Large language models contribute to planning and simulation in
AD, particularly through their proficiency in reasoning, code generation and
translation. In parallel, vision foundation models are increasingly adapted for
critical tasks such as 3D object detection and tracking, as well as creating
realistic driving scenarios for simulation and testing. Multi-modal foundation
models, integrating diverse inputs, exhibit exceptional visual understanding
and spatial reasoning, crucial for end-to-end AD. This survey not only provides
a structured taxonomy, categorizing foundation models based on their modalities
and functionalities within the AD domain but also delves into the methods
employed in current research. It identifies the gaps between existing
foundation models and cutting-edge AD approaches, thereby charting future
research directions and proposing a roadmap for bridging these gaps.