CFRet-DVQA: Coarse-to-Fine Retrieval and Efficient Tuning for Document
  Visual Question Answering
Jinxu Zhang Yongqi Yu Yu Zhang
Document Visual Question Answering (DVQA) is a task that involves responding
to queries based on the content of images. Existing work is limited to locating
information within a single page and does not facilitate cross-page
question-and-answer interaction. Furthermore, the token length limitation
imposed on inputs to the model may lead to truncation of segments pertinent to
the answer. In this study, we introduce a simple but effective methodology
called CFRet-DVQA, which focuses on retrieval and efficient tuning to address
this critical issue effectively. For that, we initially retrieve multiple
segments from the document that correlate with the question at hand.
Subsequently, we leverage the advanced reasoning abilities of the large
language model (LLM), further augmenting its performance through instruction
tuning. This approach enables the generation of answers that align with the
style of the document labels. The experiments demonstrate that our methodology
achieved state-of-the-art or competitive results with both single-page and
multi-page documents in various fields.