Model Analysis & Evaluation for Ambiguous Question Answering
Konstantinos Papakostas Irene Papadopoulou
Ambiguous questions are a challenge for Question Answering models, as they
require answers that cover multiple interpretations of the original query. To
this end, these models are required to generate long-form answers that often
combine conflicting pieces of information. Although recent advances in the
field have shown strong capabilities in generating fluent responses, certain
research questions remain unanswered. Does model/data scaling improve the
answers' quality? Do automated metrics align with human judgment? To what
extent do these models ground their answers in evidence? In this study, we aim
to thoroughly investigate these aspects, and provide valuable insights into the
limitations of the current approaches. To aid in reproducibility and further
extension of our work, we open-source our code at
https://github.com/din0s/ambig_lfqa.