Chain-of-Discussion: A Multi-Model Framework for Complex Evidence-Based
  Question Answering
Mingxu Tao Dongyan Zhao Yansong Feng
Open-ended question answering requires models to find appropriate evidence to
form well-reasoned, comprehensive and helpful answers. In practical
applications, models also need to engage in extended discussions on potential
scenarios closely relevant to the question. With augmentation of retrieval
module, open-source Large Language Models (LLMs) can produce coherent answers
often with different focuses, but are still sub-optimal in terms of reliable
evidence selection and in-depth question analysis. In this paper, we propose a
novel Chain-of-Discussion framework to leverage the synergy among multiple
open-source LLMs aiming to provide \textbf{more correct} and \textbf{more
comprehensive} answers for open-ended QA, although they are not strong enough
individually. Our experiments show that discussions among multiple LLMs play a
vital role in enhancing the quality of answers. We release our data and code at
\url{https://github.com/kobayashikanna01/Chain-of-Discussion}.