Mastering the Task of Open Information Extraction with Large Language
  Models and Consistent Reasoning Environment
Ji Qi Kaixuan Ji Xiaozhi Wang Jifan Yu Kaisheng Zeng Lei Hou Juanzi Li Bin Xu
Open Information Extraction (OIE) aims to extract objective structured
knowledge from natural texts, which has attracted growing attention to build
dedicated models with human experience. As the large language models (LLMs)
have exhibited remarkable in-context learning capabilities, a question arises
as to whether the task of OIE can be effectively tackled with this paradigm? In
this paper, we explore solving the OIE problem by constructing an appropriate
reasoning environment for LLMs. Specifically, we first propose a method to
effectively estimate the discrepancy of syntactic distribution between a LLM
and test samples, which can serve as correlation evidence for preparing
positive demonstrations. Upon the evidence, we introduce a simple yet effective
mechanism to establish the reasoning environment for LLMs on specific tasks.
Without bells and whistles, experimental results on the standard CaRB benchmark
demonstrate that our $6$-shot approach outperforms state-of-the-art supervised
method, achieving an $55.3$ $F_1$ score. Further experiments on TACRED and
ACE05 show that our method can naturally generalize to other information
extraction tasks, resulting in improvements of $5.7$ and $6.8$ $F_1$ scores,
respectively.