BlendX: Complex Multi-Intent Detection with Blended Patterns
Yejin Yoon Jungyeon Lee Kangsan Kim Chanhee Park Taeuk Kim
Task-oriented dialogue (TOD) systems are commonly designed with the
presumption that each utterance represents a single intent. However, this
assumption may not accurately reflect real-world situations, where users
frequently express multiple intents within a single utterance. While there is
an emerging interest in multi-intent detection (MID), existing in-domain
datasets such as MixATIS and MixSNIPS have limitations in their formulation. To
address these issues, we present BlendX, a suite of refined datasets featuring
more diverse patterns than their predecessors, elevating both its complexity
and diversity. For dataset construction, we utilize both rule-based heuristics
as well as a generative tool -- OpenAI's ChatGPT -- which is augmented with a
similarity-driven strategy for utterance selection. To ensure the quality of
the proposed datasets, we also introduce three novel metrics that assess the
statistical properties of an utterance related to word count, conjunction use,
and pronoun usage. Extensive experiments on BlendX reveal that state-of-the-art
MID models struggle with the challenges posed by the new datasets, highlighting
the need to reexamine the current state of the MID field. The dataset is
available at https://github.com/HYU-NLP/BlendX.