Retrieval-based Full-length Wikipedia Generation for Emergent Events
Jiebin Zhang Eugene J. Yu Qinyu Chen Chenhao Xiong Dawei Zhu Han Qian Mingbo Song Xiaoguang Li Qun Liu Sujian Li
In today's fast-paced world, the growing demand to quickly generate
comprehensive and accurate Wikipedia documents for emerging events is both
crucial and challenging. However, previous efforts in Wikipedia generation have
often fallen short of meeting real-world requirements. Some approaches focus
solely on generating segments of a complete Wikipedia document, while others
overlook the importance of faithfulness in generation or fail to consider the
influence of the pre-training corpus. In this paper, we simulate a real-world
scenario where structured full-length Wikipedia documents are generated for
emergent events using input retrieved from web sources. To ensure that Large
Language Models (LLMs) are not trained on corpora related to recently occurred
events, we select events that have taken place recently and introduce a new
benchmark Wiki-GenBen, which consists of 309 events paired with their
corresponding retrieved web pages for generating evidence. Additionally, we
design a comprehensive set of systematic evaluation metrics and baseline
methods, to evaluate the capability of LLMs in generating factual full-length
Wikipedia documents. The data and code are open-sourced at WikiGenBench.