Leveraging Explicit Procedural Instructions for Data-Efficient Action
  Prediction
Julia White Arushi Raghuvanshi Yada Pruksachatkun
Task-oriented dialogues often require agents to enact complex, multi-step
procedures in order to meet user requests. While large language models have
found success automating these dialogues in constrained environments, their
widespread deployment is limited by the substantial quantities of task-specific
data required for training. The following paper presents a data-efficient
solution to constructing dialogue systems, leveraging explicit instructions
derived from agent guidelines, such as company policies or customer service
manuals. Our proposed Knowledge-Augmented Dialogue System (KADS) combines a
large language model with a knowledge retrieval module that pulls documents
outlining relevant procedures from a predefined set of policies, given a
user-agent interaction. To train this system, we introduce a semi-supervised
pre-training scheme that employs dialogue-document matching and action-oriented
masked language modeling with partial parameter freezing. We evaluate the
effectiveness of our approach on prominent task-oriented dialogue datasets,
Action-Based Conversations Dataset and Schema-Guided Dialogue, for two dialogue
tasks: action state tracking and workflow discovery. Our results demonstrate
that procedural knowledge augmentation improves accuracy predicting in- and
out-of-distribution actions while preserving high performance in settings with
low or sparse data.