Medical Report Generation based on Segment-Enhanced Contrastive
  Representation Learning
Ruoqing Zhao Xi Wang Hongliang Dai Pan Gao Piji Li
Automated radiology report generation has the potential to improve radiology
reporting and alleviate the workload of radiologists. However, the medical
report generation task poses unique challenges due to the limited availability
of medical data and the presence of data bias. To maximize the utility of
available data and reduce data bias, we propose MSCL (Medical image
Segmentation with Contrastive Learning), a framework that utilizes the Segment
Anything Model (SAM) to segment organs, abnormalities, bones, etc., and can pay
more attention to the meaningful ROIs in the image to get better visual
representations. Then we introduce a supervised contrastive loss that assigns
more weight to reports that are semantically similar to the target while
training. The design of this loss function aims to mitigate the impact of data
bias and encourage the model to capture the essential features of a medical
image and generate high-quality reports. Experimental results demonstrate the
effectiveness of our proposed model, where we achieve state-of-the-art
performance on the IU X-Ray public dataset.