RSVP: Customer Intent Detection via Agent Response Contrastive and
  Generative Pre-Training
Yu-Chien Tang Wei-Yao Wang An-Zi Yen Wen-Chih Peng
The dialogue systems in customer services have been developed with neural
models to provide users with precise answers and round-the-clock support in
task-oriented conversations by detecting customer intents based on their
utterances. Existing intent detection approaches have highly relied on
adaptively pre-training language models with large-scale datasets, yet the
predominant cost of data collection may hinder their superiority. In addition,
they neglect the information within the conversational responses of the agents,
which have a lower collection cost, but are significant to customer intent as
agents must tailor their replies based on the customers' intent. In this paper,
we propose RSVP, a self-supervised framework dedicated to task-oriented
dialogues, which utilizes agent responses for pre-training in a two-stage
manner. Specifically, we introduce two pre-training tasks to incorporate the
relations of utterance-response pairs: 1) Response Retrieval by selecting a
correct response from a batch of candidates, and 2) Response Generation by
mimicking agents to generate the response to a given utterance. Our benchmark
results for two real-world customer service datasets show that RSVP
significantly outperforms the state-of-the-art baselines by 4.95% for accuracy,
3.4% for MRR@3, and 2.75% for MRR@5 on average. Extensive case studies are
investigated to show the validity of incorporating agent responses into the
pre-training stage.