Procedural Text Mining with Large Language Models
Anisa Rula Jennifer D'Souza
Recent advancements in the field of Natural Language Processing, particularly
the development of large-scale language models that are pretrained on vast
amounts of knowledge, are creating novel opportunities within the realm of
Knowledge Engineering. In this paper, we investigate the usage of large
language models (LLMs) in both zero-shot and in-context learning settings to
tackle the problem of extracting procedures from unstructured PDF text in an
incremental question-answering fashion. In particular, we leverage the current
state-of-the-art GPT-4 (Generative Pre-trained Transformer 4) model,
accompanied by two variations of in-context learning that involve an ontology
with definitions of procedures and steps and a limited number of samples of
few-shot learning. The findings highlight both the promise of this approach and
the value of the in-context learning customisations. These modifications have
the potential to significantly address the challenge of obtaining sufficient
training data, a hurdle often encountered in deep learning-based Natural
Language Processing techniques for procedure extraction.