What's the Plan? Evaluating and Developing Planning-Aware Techniques for
  LLMs
Eran Hirsch Guy Uziel Ateret Anaby-Tavor
Planning is a fundamental task in artificial intelligence that involves
finding a sequence of actions that achieve a specified goal in a given
environment. Large language models (LLMs) are increasingly used for
applications that require planning capabilities, such as web or embodied
agents. In line with recent studies, we demonstrate through experimentation
that LLMs lack necessary skills required for planning. Based on these
observations, we advocate for the potential of a hybrid approach that combines
LLMs with classical planning methodology. Then, we introduce SimPlan, a novel
hybrid-method, and evaluate its performance in a new challenging setup. Our
extensive experiments across various planning domains demonstrate that SimPlan
significantly outperforms existing LLM-based planners.