Retrieval-Augmented Thought Process as Sequential Decision Making
Thomas Pouplin Hao Sun Samuel Holt Mihaela van der Schaar
Large Language Models (LLMs) have demonstrated their strong ability to assist
people and show "sparks of intelligence". However, several open challenges
hinder their wider application: such as concerns over privacy, tendencies to
produce hallucinations, and difficulties in handling long contexts. In this
work, we address those challenges by introducing the Retrieval-Augmented
Thought Process (RATP). Given access to external knowledge, RATP formulates the
thought generation of LLMs as a multiple-step decision process. To optimize
such a thought process, RATP leverages Monte-Carlo Tree Search, and learns a
Q-value estimator that permits cost-efficient inference. In addressing the task
of question-answering with private data, where ethical and security concerns
limit LLM training methods, RATP achieves a 50% improvement over existing
in-context retrieval-augmented language models.