Going beyond research datasets: Novel intent discovery in the industry
  setting
Aleksandra Chrabrowa Tsimur Hadeliya Dariusz Kajtoch Robert Mroczkowski Piotr Rybak
Novel intent discovery automates the process of grouping similar messages
(questions) to identify previously unknown intents. However, current research
focuses on publicly available datasets which have only the question field and
significantly differ from real-life datasets. This paper proposes methods to
improve the intent discovery pipeline deployed in a large e-commerce platform.
We show the benefit of pre-training language models on in-domain data: both
self-supervised and with weak supervision. We also devise the best method to
utilize the conversational structure (i.e., question and answer) of real-life
datasets during fine-tuning for clustering tasks, which we call Conv. All our
methods combined to fully utilize real-life datasets give up to 33pp
performance boost over state-of-the-art Constrained Deep Adaptive Clustering
(CDAC) model for question only. By comparison CDAC model for the question data
only gives only up to 13pp performance boost over the naive baseline.