Quantitative knowledge retrieval from large language models
David Selby Kai Spriestersbach Yuichiro Iwashita Dennis Bappert Archana Warrier Sumantrak Mukherjee Muhammad Nabeel Asim Koichi Kise Sebastian Vollmer
Large language models (LLMs) have been extensively studied for their
abilities to generate convincing natural language sequences, however their
utility for quantitative information retrieval is less well understood. In this
paper we explore the feasibility of LLMs as a mechanism for quantitative
knowledge retrieval to aid data analysis tasks such as elicitation of prior
distributions for Bayesian models and imputation of missing data. We present a
prompt engineering framework, treating an LLM as an interface to a latent space
of scientific literature, comparing responses in different contexts and domains
against more established approaches. Implications and challenges of using LLMs
as 'experts' are discussed.