One Agent Too Many: User Perspectives on Approaches to Multi-agent
  Conversational AI
Christopher Clarke Karthik Krishnamurthy Walter Talamonti Yiping Kang Lingjia Tang Jason Mars
Conversational agents have been gaining increasing popularity in recent
years. Influenced by the widespread adoption of task-oriented agents such as
Apple Siri and Amazon Alexa, these agents are being deployed into various
applications to enhance user experience. Although these agents promote "ask me
anything" functionality, they are typically built to focus on a single or
finite set of expertise. Given that complex tasks often require more than one
expertise, this results in the users needing to learn and adopt multiple
agents. One approach to alleviate this is to abstract the orchestration of
agents in the background. However, this removes the option of choice and
flexibility, potentially harming the ability to complete tasks. In this paper,
we explore these different interaction experiences (one agent for all) vs (user
choice of agents) for conversational AI. We design prototypes for each,
systematically evaluating their ability to facilitate task completion. Through
a series of conducted user studies, we show that users have a significant
preference for abstracting agent orchestration in both system usability and
system performance. Additionally, we demonstrate that this mode of interaction
is able to provide quality responses that are rated within 1% of human-selected
answers.