Towards LLM-driven Dialogue State Tracking
Yujie Feng Zexin Lu Bo Liu Liming Zhan Xiao-Ming Wu
Dialogue State Tracking (DST) is of paramount importance in ensuring accurate
tracking of user goals and system actions within task-oriented dialogue
systems. The emergence of large language models (LLMs) such as GPT3 and ChatGPT
has sparked considerable interest in assessing their efficacy across diverse
applications. In this study, we conduct an initial examination of ChatGPT's
capabilities in DST. Our evaluation uncovers the exceptional performance of
ChatGPT in this task, offering valuable insights to researchers regarding its
capabilities and providing useful directions for designing and enhancing
dialogue systems. Despite its impressive performance, ChatGPT has significant
limitations including its closed-source nature, request restrictions, raising
data privacy concerns, and lacking local deployment capabilities. To address
these concerns, we present LDST, an LLM-driven DST framework based on smaller,
open-source foundation models. By utilizing a novel domain-slot instruction
tuning method, LDST achieves performance on par with ChatGPT. Comprehensive
evaluations across three distinct experimental settings, we find that LDST
exhibits remarkable performance improvements in both zero-shot and few-shot
setting compared to previous SOTA methods. The source code is provided for
reproducibility.