Promoting Open-domain Dialogue Generation through Learning Pattern
  Information between Contexts and Responses
Mengjuan Liu Chenyang Liu Yunfan Yang Jiang Liu Mohan Jing
Recently, utilizing deep neural networks to build the opendomain dialogue
models has become a hot topic. However, the responses generated by these models
suffer from many problems such as responses not being contextualized and tend
to generate generic responses that lack information content, damaging the
user's experience seriously. Therefore, many studies try introducing more
information into the dialogue models to make the generated responses more vivid
and informative. Unlike them, this paper improves the quality of generated
responses by learning the implicit pattern information between contexts and
responses in the training samples. In this paper, we first build an open-domain
dialogue model based on the pre-trained language model (i.e., GPT-2). And then,
an improved scheduled sampling method is proposed for pre-trained models, by
which the responses can be used to guide the response generation in the
training phase while avoiding the exposure bias problem. More importantly, we
design a response-aware mechanism for mining the implicit pattern information
between contexts and responses so that the generated replies are more diverse
and approximate to human replies. Finally, we evaluate the proposed model (RAD)
on the Persona-Chat and DailyDialog datasets; and the experimental results show
that our model outperforms the baselines on most automatic and manual metrics.