Replace and Report: NLP Assisted Radiology Report Generation
Kaveri Kale pushpak Bhattacharyya Kshitij Jadhav
Clinical practice frequently uses medical imaging for diagnosis and
treatment. A significant challenge for automatic radiology report generation is
that the radiology reports are long narratives consisting of multiple sentences
for both abnormal and normal findings. Therefore, applying conventional image
captioning approaches to generate the whole report proves to be insufficient,
as these are designed to briefly describe images with short sentences. We
propose a template-based approach to generate radiology reports from
radiographs. Our approach involves the following: i) using a multilabel image
classifier, produce the tags for the input radiograph; ii) using a
transformer-based model, generate pathological descriptions (a description of
abnormal findings seen on radiographs) from the tags generated in step (i);
iii) using a BERT-based multi-label text classifier, find the spans in the
normal report template to replace with the generated pathological descriptions;
and iv) using a rule-based system, replace the identified span with the
generated pathological description. We performed experiments with the two most
popular radiology report datasets, IU Chest X-ray and MIMIC-CXR and
demonstrated that the BLEU-1, ROUGE-L, METEOR, and CIDEr scores are better than
the State-of-the-Art models by 25%, 36%, 44% and 48% respectively, on the IU
X-RAY dataset. To the best of our knowledge, this is the first attempt to
generate chest X-ray radiology reports by first creating small sentences for
abnormal findings and then replacing them in the normal report template.