AutoGuide: Automated Generation and Selection of State-Aware Guidelines
  for Large Language Model Agents
Yao Fu Dong-Ki Kim Jaekyeom Kim Sungryull Sohn Lajanugen Logeswaran Kyunghoon Bae Honglak Lee
The primary limitation of large language models (LLMs) is their restricted
understanding of the world. This poses significant difficulties for LLM-based
agents, particularly in domains where pre-trained LLMs lack sufficient
knowledge. In this paper, we introduce a novel framework, called AutoGuide,
that bridges the knowledge gap in pre-trained LLMs by leveraging implicit
knowledge in offline experiences. Specifically, AutoGuide effectively extracts
knowledge embedded in offline data by extracting a set of state-aware
guidelines. Importantly, each state-aware guideline is expressed in concise
natural language and follows a conditional structure, clearly describing the
state where it is applicable. As such, the resulting guidelines enable a
principled way to provide helpful knowledge pertinent to an agent's current
decision-making process. We show that our approach outperforms competitive
LLM-based baselines by a large margin in sequential decision-making benchmarks.