SocraSynth: Multi-LLM Reasoning with Conditional Statistics
Edward Y. Chang
Large language models (LLMs), while promising, face criticisms for biases,
hallucinations, and a lack of reasoning capability. This paper introduces
SocraSynth, a multi-LLM agent reasoning platform developed to mitigate these
issues. SocraSynth utilizes conditional statistics and systematic context
enhancement through continuous arguments, alongside adjustable debate
contentiousness levels. The platform typically involves a human moderator and
two LLM agents representing opposing viewpoints on a given subject. SocraSynth
operates in two main phases: knowledge generation and reasoning evaluation. In
the knowledge generation phase, the moderator defines the debate topic and
contentiousness level, prompting the agents to formulate supporting arguments
for their respective stances. The reasoning evaluation phase then employs
Socratic reasoning and formal logic principles to appraise the quality of the
arguments presented. The dialogue concludes with the moderator adjusting the
contentiousness from confrontational to collaborative, gathering final,
conciliatory remarks to aid in human reasoning and decision-making. Through
case studies in three distinct application domains, this paper showcases
SocraSynth's effectiveness in fostering rigorous research, dynamic reasoning,
comprehensive assessment, and enhanced collaboration. This underscores the
value of multi-agent interactions in leveraging LLMs for advanced knowledge
extraction and decision-making support.