WordScape: a Pipeline to extract multilingual, visually rich Documents
  with Layout Annotations from Web Crawl Data
Maurice Weber Carlo Siebenschuh Rory Butler Anton Alexandrov Valdemar Thanner Georgios Tsolakis Haris Jabbar Ian Foster Bo Li Rick Stevens Ce Zhang
We introduce WordScape, a novel pipeline for the creation of
cross-disciplinary, multilingual corpora comprising millions of pages with
annotations for document layout detection. Relating visual and textual items on
document pages has gained further significance with the advent of multimodal
models. Various approaches proved effective for visual question answering or
layout segmentation. However, the interplay of text, tables, and visuals
remains challenging for a variety of document understanding tasks. In
particular, many models fail to generalize well to diverse domains and new
languages due to insufficient availability of training data. WordScape
addresses these limitations. Our automatic annotation pipeline parses the Open
XML structure of Word documents obtained from the web, jointly providing
layout-annotated document images and their textual representations. In turn,
WordScape offers unique properties as it (1) leverages the ubiquity of the Word
file format on the internet, (2) is readily accessible through the Common Crawl
web corpus, (3) is adaptive to domain-specific documents, and (4) offers
culturally and linguistically diverse document pages with natural semantic
structure and high-quality text. Together with the pipeline, we will
additionally release 9.5M urls to word documents which can be processed using
WordScape to create a dataset of over 40M pages. Finally, we investigate the
quality of text and layout annotations extracted by WordScape, assess the
impact on document understanding benchmarks, and demonstrate that manual
labeling costs can be substantially reduced.