Can Prompt Learning Benefit Radiology Report Generation?
Jun Wang Lixing Zhu Abhir Bhalerao Yulan He
Radiology report generation aims to automatically provide clinically
meaningful descriptions of radiology images such as MRI and X-ray. Although
great success has been achieved in natural scene image captioning tasks,
radiology report generation remains challenging and requires prior medical
knowledge. In this paper, we propose PromptRRG, a method that utilizes prompt
learning to activate a pretrained model and incorporate prior knowledge. Since
prompt learning for radiology report generation has not been explored before,
we begin with investigating prompt designs and categorise them based on varying
levels of knowledge: common, domain-specific and disease-enriched prompts.
Additionally, we propose an automatic prompt learning mechanism to alleviate
the burden of manual prompt engineering. This is the first work to
systematically examine the effectiveness of prompt learning for radiology
report generation. Experimental results on the largest radiology report
generation benchmark, MIMIC-CXR, demonstrate that our proposed method achieves
state-of-the-art performance. Code will be available upon the acceptance.