MFORT-QA: Multi-hop Few-shot Open Rich Table Question Answering
Che Guan Mengyu Huang Peng Zhang
In today's fast-paced industry, professionals face the challenge of
summarizing a large number of documents and extracting vital information from
them on a daily basis. These metrics are frequently hidden away in tables
and/or their nested hyperlinks. To address this challenge, the approach of
Table Question Answering (QA) has been developed to extract the relevant
information. However, traditional Table QA training tasks that provide a table
and an answer(s) from a gold cell coordinate(s) for a question may not always
ensure extracting the accurate answer(s). Recent advancements in Large Language
Models (LLMs) have opened up new possibilities for extracting information from
tabular data using prompts. In this paper, we introduce the Multi-hop Few-shot
Open Rich Table QA (MFORT-QA) approach, which consists of two major steps. The
first step involves Few-Shot Learning (FSL), where relevant tables and
associated contexts of hyperlinks are retrieved based on a given question. The
retrieved content is then used to construct few-shot prompts as inputs to an
LLM, such as ChatGPT. To tackle the challenge of answering complex questions,
the second step leverages Chain-of-thought (CoT) prompting to decompose the
complex question into a sequential chain of questions and reasoning thoughts in
a multi-hop manner. Retrieval-Augmented Generation (RAG) enhances this process
by retrieving relevant tables and contexts of hyperlinks that are relevant to
the resulting reasoning thoughts and questions. These additional contexts are
then used to supplement the prompt used in the first step, resulting in more
accurate answers from an LLM. Empirical results from OTT-QA demonstrate that
our abstractive QA approach significantly improves the accuracy of extractive
Table QA methods.