BEV-CLIP: Multi-modal BEV Retrieval Methodology for Complex Scene in
  Autonomous Driving
Dafeng Wei Tian Gao Zhengyu Jia Changwei Cai Chengkai Hou Peng Jia Fu Liu Kun Zhan Jingchen Fan Yixing Zhao Yang Wang
The demand for the retrieval of complex scene data in autonomous driving is
increasing, especially as passenger vehicles have been equipped with the
ability to navigate urban settings, with the imperative to address long-tail
scenarios. Meanwhile, under the pre-existing two dimensional image retrieval
method, some problems may arise with scene retrieval, such as lack of global
feature representation and subpar text retrieval ability. To address these
issues, we have proposed \textbf{BEV-CLIP}, the first multimodal Bird's-Eye
View(BEV) retrieval methodology that utilizes descriptive text as an input to
retrieve corresponding scenes. This methodology applies the semantic feature
extraction abilities of a large language model (LLM) to facilitate zero-shot
retrieval of extensive text descriptions, and incorporates semi-structured
information from a knowledge graph to improve the semantic richness and variety
of the language embedding. Our experiments result in 87.66% accuracy on
NuScenes dataset in text-to-BEV feature retrieval. The demonstrated cases in
our paper support that our retrieval method is also indicated to be effective
in identifying certain long-tail corner scenes.