Large Language Models Powered Context-aware Motion Prediction
Xiaoji Zheng Lixiu Wu Zhijie Yan Yuanrong Tang Hao Zhao Chen Zhong Bokui Chen Jiangtao Gong
Motion prediction is among the most fundamental tasks in autonomous driving.
Traditional methods of motion forecasting primarily encode vector information
of maps and historical trajectory data of traffic participants, lacking a
comprehensive understanding of overall traffic semantics, which in turn affects
the performance of prediction tasks. In this paper, we utilized Large Language
Models (LLMs) to enhance the global traffic context understanding for motion
prediction tasks. We first conducted systematic prompt engineering, visualizing
complex traffic environments and historical trajectory information of traffic
participants into image prompts -- Transportation Context Map (TC-Map),
accompanied by corresponding text prompts. Through this approach, we obtained
rich traffic context information from the LLM. By integrating this information
into the motion prediction model, we demonstrate that such context can enhance
the accuracy of motion predictions. Furthermore, considering the cost
associated with LLMs, we propose a cost-effective deployment strategy:
enhancing the accuracy of motion prediction tasks at scale with 0.7\%
LLM-augmented datasets. Our research offers valuable insights into enhancing
the understanding of traffic scenes of LLMs and the motion prediction
performance of autonomous driving.