Navigation with Large Language Models: Semantic Guesswork as a Heuristic
  for Planning
Dhruv Shah Michael Equi Blazej Osinski Fei Xia Brian Ichter Sergey Levine
Navigation in unfamiliar environments presents a major challenge for robots:
while mapping and planning techniques can be used to build up a representation
of the world, quickly discovering a path to a desired goal in unfamiliar
settings with such methods often requires lengthy mapping and exploration.
Humans can rapidly navigate new environments, particularly indoor environments
that are laid out logically, by leveraging semantics -- e.g., a kitchen often
adjoins a living room, an exit sign indicates the way out, and so forth.
Language models can provide robots with such knowledge, but directly using
language models to instruct a robot how to reach some destination can also be
impractical: while language models might produce a narrative about how to reach
some goal, because they are not grounded in real-world observations, this
narrative might be arbitrarily wrong. Therefore, in this paper we study how the
``semantic guesswork'' produced by language models can be utilized as a guiding
heuristic for planning algorithms. Our method, Language Frontier Guide (LFG),
uses the language model to bias exploration of novel real-world environments by
incorporating the semantic knowledge stored in language models as a search
heuristic for planning with either topological or metric maps. We evaluate LFG
in challenging real-world environments and simulated benchmarks, outperforming
uninformed exploration and other ways of using language models.