Learning to Select the Relevant History Turns in Conversational Question
  Answering
Munazza Zaib Wei Emma Zhang Quan Z. Sheng Subhash Sagar Adnan Mahmood Yang Zhang
The increasing demand for the web-based digital assistants has given a rapid
rise in the interest of the Information Retrieval (IR) community towards the
field of conversational question answering (ConvQA). However, one of the
critical aspects of ConvQA is the effective selection of conversational history
turns to answer the question at hand. The dependency between relevant history
selection and correct answer prediction is an intriguing but under-explored
area. The selected relevant context can better guide the system so as to where
exactly in the passage to look for an answer. Irrelevant context, on the other
hand, brings noise to the system, thereby resulting in a decline in the model's
performance. In this paper, we propose a framework, DHS-ConvQA (Dynamic History
Selection in Conversational Question Answering), that first generates the
context and question entities for all the history turns, which are then pruned
on the basis of similarity they share in common with the question at hand. We
also propose an attention-based mechanism to re-rank the pruned terms based on
their calculated weights of how useful they are in answering the question. In
the end, we further aid the model by highlighting the terms in the re-ranked
conversational history using a binary classification task and keeping the
useful terms (predicted as 1) and ignoring the irrelevant terms (predicted as
0). We demonstrate the efficacy of our proposed framework with extensive
experimental results on CANARD and QuAC -- the two popularly utilized datasets
in ConvQA. We demonstrate that selecting relevant turns works better than
rewriting the original question. We also investigate how adding the irrelevant
history turns negatively impacts the model's performance and discuss the
research challenges that demand more attention from the IR community.