LiveChat: A Large-Scale Personalized Dialogue Dataset Automatically
  Constructed from Live Streaming
Jingsheng Gao Yixin Lian Ziyi Zhou Yuzhuo Fu Baoyuan Wang
Open-domain dialogue systems have made promising progress in recent years.
While the state-of-the-art dialogue agents are built upon large-scale
text-based social media data and large pre-trained models, there is no
guarantee these agents could also perform well in fast-growing scenarios, such
as live streaming, due to the bounded transferability of pre-trained models and
biased distributions of public datasets from Reddit and Weibo, etc. To improve
the essential capability of responding and establish a benchmark in the live
open-domain scenario, we introduce the LiveChat dataset, composed of 1.33
million real-life Chinese dialogues with almost 3800 average sessions across
351 personas and fine-grained profiles for each persona. LiveChat is
automatically constructed by processing numerous live videos on the Internet
and naturally falls within the scope of multi-party conversations, where the
issues of Who says What to Whom should be considered. Therefore, we target two
critical tasks of response modeling and addressee recognition and propose
retrieval-based baselines grounded on advanced techniques. Experimental results
have validated the positive effects of leveraging persona profiles and larger
average sessions per persona. In addition, we also benchmark the
transferability of advanced generation-based models on LiveChat and pose some
future directions for current challenges.