Parrot: Enhancing Multi-Turn Chat Models by Learning to Ask Questions
Yuchong Sun Che Liu Jinwen Huang Ruihua Song Fuzheng Zhang Di Zhang Zhongyuan Wang Kun Gai
Impressive progress has been made on chat models based on Large Language
Models (LLMs) recently; however, there is a noticeable lag in multi-turn
conversations between open-source chat models (e.g., Alpaca and Vicuna) and the
leading chat models (e.g., ChatGPT and GPT-4). Through a series of analyses, we
attribute the lag to the lack of enough high-quality multi-turn
instruction-tuning data. The available instruction-tuning data for the
community are either single-turn conversations or multi-turn ones with certain
issues, such as non-human-like instructions, less detailed responses, or rare
topic shifts. In this paper, we address these challenges by introducing Parrot,
a highly scalable solution designed to automatically generate high-quality
instruction-tuning data, which are then used to enhance the effectiveness of
chat models in multi-turn conversations. Specifically, we start by training the
Parrot-Ask model, which is designed to emulate real users in generating
instructions. We then utilize Parrot-Ask to engage in multi-turn conversations
with ChatGPT across a diverse range of topics, resulting in a collection of 40K
high-quality multi-turn dialogues (Parrot-40K). These data are subsequently
employed to train a chat model that we have named Parrot-Chat. We demonstrate
that the dialogues gathered from Parrot-Ask markedly outperform existing
multi-turn instruction-following datasets in critical metrics, including topic
diversity, number of turns, and resemblance to human conversation. With only
40K training examples, Parrot-Chat achieves strong performance against other
13B open-source models across a range of instruction-following benchmarks, and
particularly excels in evaluations of multi-turn capabilities. We make all
codes, datasets, and two versions of the Parrot-Ask model based on LLaMA2-13B
and KuaiYii-13B available at https://github.com/kwai/KwaiYii/Parrot.