Conventional methods model open domain dialogue generation as a black box through end-to-end learning from large scale conversation data. In this work, we make the first step to open the black box by introducing dialogue acts into open domain dialogue generation. The dialogue acts are generally designed and reveal how people engage in social chat. Inspired by analysis on real data, we propose jointly modeling dialogue act selection and response generation, and perform learning with human-human conversations tagged with a dialogue act classifier and a reinforcement approach to further optimizing the model for long-term conversation. With the dialogue acts, we not only achieve significant improvement over state-of-the-art methods on response quality for given contexts and long-term conversation in both machine-machine simulation and human-machine conversation, but also are capable of explaining why such achievements can be made. Conversational agents are becoming ubiquitous recently. Through human-machine conversation, such agents either help users complete specific tasks BID39 or engage them in social chat BID30 . Depending on application scenarios, various conversational agents have been designed including chatbots, personal assistants, and automated customer service, etc.Traditional research on conversational agents focuses on task-oriented dialogue systems BID39 where task specific dialogue acts are handcrafted in a form of slot-value pairs. On the one hand, through slot-filling, the dialogue acts make conversations in such systems interpretable and controllable; on the other hand, they also hinder scaling such systems to new domains. To escape from the limitation, recent interest of research moves to end-to-end dialogue learning without any assumptions on dialogue acts. Most of the effort is paid to non-task-oriented chit-chat BID30 , and there are also a few studies on task-oriented dialogues BID1 BID4 . Without dialogue acts, these work directly constructs a response by learning from large scale data with neural networks, and thus is easy to scale to new domains. On the other hand, due to the absence of dialogue acts, it is hard to interpret the emergence of a response to a dialogue context and predict where the conversation will flow to.In this work, we aim to achieve interpretability and controllability in non-task-oriented dialogues. To this end, we introduce dialogue acts into open domain dialogue generation. Open domain dialogue generation has been widely applied to chatbots which aim at engaging users by keeping conversation going. Existing work concentrates on generating relevant and diverse responses for a static context. However, it is not clear if relevance and diversity are sufficient to engagement in dynamic interactions. Therefore, we investigate the following problems: (1) if we can properly design dialogue acts that can enable us to understand engagement in human-human open domain conversation; (2) how to learn a dialogue generation model with the dialogue acts; and (3) how the model performs in practice and if the performance can be explained by the dialogue acts.To examine how people engage in social chat, we establish a general dialogue act taxonomy for open domain conversation by extending the existing work with high-level dialogue acts regarding to conversational context. The taxonomy, when applied to real data, gives rise to an interesting finding that in addition to replying with relevance and diversity , people are used to driving their social chat by constantly switching to new contexts and properly asking questions. Such behaviors are less explored before, and thus are difficult for the existing end-to-end learning methods to imitate. To mimic human behaviors, we propose jointly modeling dialogue act selection and response generation in open domain dialogue generation. The dialogue model is specified with neural networks. We propose learning from human-human interactions by fitting the model to large scale real world dialogues tagged with a dialogue act classifier and further optimizing the policy of act selection for long-term conversation through a reinforcement learning approach. Our model enjoys several advantages over the existing models: (1) the dialogue acts provide interpretation to response generation from a discourse perspective; (2) the dialogue acts enhance diversity of responses by expanding the search space from language to act × language; (3) the dialogue acts manage the flow of humanmachine conversations and thus enhance human engagement; and (4) the dialogue act selection is compatible with post-engineering work (e.g., combination with rules), and thus allows engineers to flexibly control their systems through picking responses from their desired dialogue acts. Evaluation results on large scale test data indicate that our model can significantly outperform state-of-the-art methods in terms of quality of generated responses regarding to given contexts and lead to longterm conversation in both machine-machine simulation and human-machine conversation in a way similar to how human behave in their interactions.Our contributions in this work include: (1) design of dialogue acts that represent human behavior regarding to conversational context and insights from analysis of human-human interactions with the design; (2) joint modeling of dialogue act selection and response generation in open domain dialogue generation; (3) proposal of a supervised learning approach and a reinforcement learning approach for model optimization; (4) empirical verification of the effectiveness of the model through automatic metrics, human annotations, machine-machine simulation, and human-machine conversation. A response or an answer to the previous utterances in the current context. "this summer." after "when are you going to Tokyo?". Finally, we study how the generated responses are affected by the dialogue acts. We collect generated responses from a specific dialogue act for the contexts of the test dialogues, and characterize the responses with the following metrics: (1) distinct-1 and distinct-2; (2) words out of context (OOC): ratio of words that are in the generated responses but not contained by the contexts; and (3) average length of the generated responses (Ave Len). TAB9 reports the results 4 . In general, responses generated from CS. * are longer, more informative, and contain more new words than responses generated from CM. *, which has been illustrated by the example in TAB7 . Another interesting finding is that statements and answers are generally more informative than questions in both CS. * and CM.*. In addition to these metrics, we also calculate BLEU scores and embedding based metrics, but do not observe significant difference among responses from different dialogue acts. The reason might be that these metrics are based on comparsion of the generated responses and human responses, but human responses in the test set are inherently mixture of responses from different dialogue acts. We study open domain dialogue generation with generally designed dialogue acts that can describe human behavior in social interactions. To mimic such behavior, we propose jointly modeling dialogue act selection and response generation, and perform both supervised learning with a learned dialogue act classifier and reinforcement learning for long-term conversation. Empirical studies on response generation for given contexts, machine-machine simulation, and human-machine conversation show that the proposed models can significantly outperform state-of-the-art methods.
open domain dialogue generation with dialogue acts The authors use a distant supervision technique to add dialogue act tags as a conditioning factor for generating responses in open-domain dialogues The paper describes a technique to incorporate dialog acts into neural conversational agents