From RAGs to riches: Using large language models to write documents for
  clinical trials
Nigel Markey Ilyass El-Mansouri Gaetan Rensonnet Casper van Langen Christoph Meier
Clinical trials require numerous documents to be written -- protocols,
consent forms, clinical study reports and others. Large language models (LLMs)
offer the potential to rapidly generate first versions of these documents,
however there are concerns about the quality of their output Here we report an
evaluation of LLMs in generating parts of one such document, clinical trial
protocols. We find that an offthe-shelf LLM delivers reasonable results,
especially when assessing content relevance and the correct use of terminology.
However, deficiencies remain: specifically clinical thinking and logic, and
appropriate use of references. To improve performance, we used
retrieval-augmented generation (RAG) to prompt an LLM with accurate up-to-date
information. As a result of using RAG, the writing quality of the LLM improves
substantially, which has implications for the practical useability of LLMs in
clinical trial-related writing.