DriveCoT: Integrating Chain-of-Thought Reasoning with End-to-End Driving
Tianqi Wang Enze Xie Ruihang Chu Zhenguo Li Ping Luo
End-to-end driving has made significant progress in recent years,
demonstrating benefits such as system simplicity and competitive driving
performance under both open-loop and closed-loop settings. Nevertheless, the
lack of interpretability and controllability in its driving decisions hinders
real-world deployment for end-to-end driving systems. In this paper, we collect
a comprehensive end-to-end driving dataset named DriveCoT, leveraging the CARLA
simulator. It contains sensor data, control decisions, and chain-of-thought
labels to indicate the reasoning process. We utilize the challenging driving
scenarios from the CARLA leaderboard 2.0, which involve high-speed driving and
lane-changing, and propose a rule-based expert policy to control the vehicle
and generate ground truth labels for its reasoning process across different
driving aspects and the final decisions. This dataset can serve as an open-loop
end-to-end driving benchmark, enabling the evaluation of accuracy in various
chain-of-thought aspects and the final decision. In addition, we propose a
baseline model called DriveCoT-Agent, trained on our dataset, to generate
chain-of-thought predictions and final decisions. The trained model exhibits
strong performance in both open-loop and closed-loop evaluations, demonstrating
the effectiveness of our proposed dataset.