LLM Guided Inductive Inference for Solving Compositional Problems
Abhigya Sodani Lauren Moos Matthew Mirman
While large language models (LLMs) have demonstrated impressive performance
in question-answering tasks, their performance is limited when the questions
require knowledge that is not included in the model's training data and can
only be acquired through direct observation or interaction with the real world.
Existing methods decompose reasoning tasks through the use of modules invoked
sequentially, limiting their ability to answer deep reasoning tasks. We
introduce a method, Recursion based extensible LLM (REBEL), which handles
open-world, deep reasoning tasks by employing automated reasoning techniques
like dynamic planning and forward-chaining strategies. REBEL allows LLMs to
reason via recursive problem decomposition and utilization of external tools.
The tools that REBEL uses are specified only by natural language description.
We further demonstrate REBEL capabilities on a set of problems that require a
deeply nested use of external tools in a compositional and conversational
setting.