Are Large Language Models Table-based Fact-Checkers?
Hangwen Zhang Qingyi Si Peng Fu Zheng Lin Weiping Wang
Table-based Fact Verification (TFV) aims to extract the entailment relation
between statements and structured tables. Existing TFV methods based on
small-scaled models suffer from insufficient labeled data and weak zero-shot
ability. Recently, the appearance of Large Language Models (LLMs) has gained
lots of attraction in research fields. They have shown powerful zero-shot and
in-context learning abilities on several NLP tasks, but their potential on TFV
is still unknown. In this work, we implement a preliminary study about whether
LLMs are table-based fact-checkers. In detail, we design diverse prompts to
explore how the in-context learning can help LLMs in TFV, i.e., zero-shot and
few-shot TFV capability. Besides, we carefully design and construct TFV
instructions to study the performance gain brought by the instruction tuning of
LLMs. Experimental results demonstrate that LLMs can achieve acceptable results
on zero-shot and few-shot TFV with prompt engineering, while instruction-tuning
can stimulate the TFV capability significantly. We also make some valuable
findings about the format of zero-shot prompts and the number of in-context
examples. Finally, we analyze some possible directions to promote the accuracy
of TFV via LLMs, which is beneficial to further research of table reasoning.