CLIP-Loc: Multi-modal Landmark Association for Global Localization in
  Object-based Maps
Shigemichi Matsuzaki Takuma Sugino Kazuhito Tanaka Zijun Sha Shintaro Nakaoka Shintaro Yoshizawa Kazuhiro Shintani
This paper describes a multi-modal data association method for global
localization using object-based maps and camera images. In global localization,
or relocalization, using object-based maps, existing methods typically resort
to matching all possible combinations of detected objects and landmarks with
the same object category, followed by inlier extraction using RANSAC or
brute-force search. This approach becomes infeasible as the number of landmarks
increases due to the exponential growth of correspondence candidates. In this
paper, we propose labeling landmarks with natural language descriptions and
extracting correspondences based on conceptual similarity with image
observations using a Vision Language Model (VLM). By leveraging detailed text
information, our approach efficiently extracts correspondences compared to
methods using only object categories. Through experiments, we demonstrate that
the proposed method enables more accurate global localization with fewer
iterations compared to baseline methods, exhibiting its efficiency.