Enhancing Pipeline-Based Conversational Agents with Large Language
  Models
Mina Foosherian Hendrik Purwins Purna Rathnayake Touhidul Alam Rui Teimao Klaus-Dieter Thoben
The latest advancements in AI and deep learning have led to a breakthrough in
large language model (LLM)-based agents such as GPT-4. However, many commercial
conversational agent development tools are pipeline-based and have limitations
in holding a human-like conversation. This paper investigates the capabilities
of LLMs to enhance pipeline-based conversational agents during two phases: 1)
in the design and development phase and 2) during operations. In 1) LLMs can
aid in generating training data, extracting entities and synonyms,
localization, and persona design. In 2) LLMs can assist in contextualization,
intent classification to prevent conversational breakdown and handle
out-of-scope questions, auto-correcting utterances, rephrasing responses,
formulating disambiguation questions, summarization, and enabling closed
question-answering capabilities. We conducted informal experiments with GPT-4
in the private banking domain to demonstrate the scenarios above with a
practical example. Companies may be hesitant to replace their pipeline-based
agents with LLMs entirely due to privacy concerns and the need for deep
integration within their existing ecosystems. A hybrid approach in which LLMs'
are integrated into the pipeline-based agents allows them to save time and
costs of building and running agents by capitalizing on the capabilities of
LLMs while retaining the integration and privacy safeguards of their existing
systems.