Context Retrieval via Normalized Contextual Latent Interaction for
  Conversational Agent
Junfeng Liu Zhuocheng Mei Kewen Peng Ranga Raju Vatsavai
Conversational agents leveraging AI, particularly deep learning, are emerging
in both academic research and real-world applications. However, these
applications still face challenges, including disrespecting knowledge and
facts, not personalizing to user preferences, and enormous demand for
computational resources during training and inference. Recent research efforts
have been focused on addressing these challenges from various aspects,
including supplementing various types of auxiliary information to the
conversational agents. However, existing methods are still not able to
effectively and efficiently exploit relevant information from these auxiliary
supplements to further unleash the power of the conversational agents and the
language models they use. In this paper, we present a novel method, PK-NCLI,
that is able to accurately and efficiently identify relevant auxiliary
information to improve the quality of conversational responses by learning the
relevance among persona, chat history, and knowledge background through
low-level normalized contextual latent interaction. Our experimental results
indicate that PK-NCLI outperforms the state-of-the-art method, PK-FoCus, by
47.80%/30.61%/24.14% in terms of perplexity, knowledge grounding, and training
efficiency, respectively, and maintained the same level of persona grounding
performance. We also provide a detailed analysis of how different factors,
including language model choices and trade-offs on training weights, would
affect the performance of PK-NCLI.