Are Large Language Models Geospatially Knowledgeable?
Prabin Bhandari Antonios Anastasopoulos Dieter Pfoser
Despite the impressive performance of Large Language Models (LLM) for various
natural language processing tasks, little is known about their comprehension of
geographic data and related ability to facilitate informed geospatial
decision-making. This paper investigates the extent of geospatial knowledge,
awareness, and reasoning abilities encoded within such pretrained LLMs. With a
focus on autoregressive language models, we devise experimental approaches
related to (i) probing LLMs for geo-coordinates to assess geospatial knowledge,
(ii) using geospatial and non-geospatial prepositions to gauge their geospatial
awareness, and (iii) utilizing a multidimensional scaling (MDS) experiment to
assess the models' geospatial reasoning capabilities and to determine locations
of cities based on prompting. Our results confirm that it does not only take
larger, but also more sophisticated LLMs to synthesize geospatial knowledge
from textual information. As such, this research contributes to understanding
the potential and limitations of LLMs in dealing with geospatial information.