Retrieval Augmented Chest X-Ray Report Generation using OpenAI GPT
  models
Mercy Ranjit Gopinath Ganapathy Ranjit Manuel Tanuja Ganu
We propose Retrieval Augmented Generation (RAG) as an approach for automated
radiology report writing that leverages multimodally aligned embeddings from a
contrastively pretrained vision language model for retrieval of relevant
candidate radiology text for an input radiology image and a general domain
generative model like OpenAI text-davinci-003, gpt-3.5-turbo and gpt-4 for
report generation using the relevant radiology text retrieved. This approach
keeps hallucinated generations under check and provides capabilities to
generate report content in the format we desire leveraging the instruction
following capabilities of these generative models. Our approach achieves better
clinical metrics with a BERTScore of 0.2865 ({\Delta}+ 25.88%) and Semb score
of 0.4026 ({\Delta}+ 6.31%). Our approach can be broadly relevant for different
clinical settings as it allows to augment the automated radiology report
generation process with content relevant for that setting while also having the
ability to inject user intents and requirements in the prompts as part of the
report generation process to modulate the content and format of the generated
reports as applicable for that clinical setting.