Adaptive-Solver Framework for Dynamic Strategy Selection in Large
  Language Model Reasoning
Jianpeng Zhou Wanjun Zhong Yanlin Wang Jiahai Wang
Large Language Models (LLMs) are showcasing impressive ability in handling
complex reasoning tasks. In real-world situations, problems often span a
spectrum of complexities. Humans inherently adjust their problem-solving
approaches based on task complexity. However, most methodologies that leverage
LLMs tend to adopt a uniform approach: utilizing consistent models, prompting
methods, and degrees of problem decomposition, regardless of the problem
complexity. Inflexibility of them can bring unnecessary computational overhead
or sub-optimal performance. To address this problem, we introduce an
Adaptive-Solver framework. It strategically modulates solving strategies based
on the difficulties of the problems. Given an initial solution, the framework
functions with two primary modules. The initial evaluation module assesses the
adequacy of the current solution. If improvements are needed, the subsequent
adaptation module comes into play. Within this module, three key adaptation
strategies are employed: (1) Model Adaptation: Switching to a stronger LLM when
a weaker variant is inadequate. (2) Prompting Method Adaptation: Alternating
between different prompting techniques to suit the problem's nuances. (3)
Decomposition Granularity Adaptation: Breaking down a complex problem into more
fine-grained sub-questions to enhance solvability. Through such dynamic
adaptations, our framework not only enhances computational efficiency but also
elevates the overall performance. This dual-benefit ensures both the efficiency
of the system for simpler tasks and the precision required for more complex
questions. Experimental results from complex reasoning tasks reveal that the
prompting method adaptation and decomposition granularity adaptation enhance
performance across all tasks. Furthermore, the model adaptation approach
significantly reduces API costs (up to 50%) while maintaining superior
performance.