A Diffusion Weighted Graph Framework for New Intent Discovery
Wenkai Shi Wenbin An Feng Tian Qinghua Zheng QianYing Wang Ping Chen
New Intent Discovery (NID) aims to recognize both new and known intents from
unlabeled data with the aid of limited labeled data containing only known
intents. Without considering structure relationships between samples, previous
methods generate noisy supervisory signals which cannot strike a balance
between quantity and quality, hindering the formation of new intent clusters
and effective transfer of the pre-training knowledge. To mitigate this
limitation, we propose a novel Diffusion Weighted Graph Framework (DWGF) to
capture both semantic similarities and structure relationships inherent in
data, enabling more sufficient and reliable supervisory signals. Specifically,
for each sample, we diffuse neighborhood relationships along semantic paths
guided by the nearest neighbors for multiple hops to characterize its local
structure discriminately. Then, we sample its positive keys and weigh them
based on semantic similarities and local structures for contrastive learning.
During inference, we further propose Graph Smoothing Filter (GSF) to explicitly
utilize the structure relationships to filter high-frequency noise embodied in
semantically ambiguous samples on the cluster boundary. Extensive experiments
show that our method outperforms state-of-the-art models on all evaluation
metrics across multiple benchmark datasets. Code and data are available at
https://github.com/yibai-shi/DWGF.