Enhanced Knowledge Injection for Radiology Report Generation
Qingqiu Li Jilan Xu Runtian Yuan Mohan Chen Yuejie Zhang Rui Feng Xiaobo Zhang Shang Gao
Automatic generation of radiology reports holds crucial clinical value, as it
can alleviate substantial workload on radiologists and remind less experienced
ones of potential anomalies. Despite the remarkable performance of various
image captioning methods in the natural image field, generating accurate
reports for medical images still faces challenges, i.e., disparities in visual
and textual data, and lack of accurate domain knowledge. To address these
issues, we propose an enhanced knowledge injection framework, which utilizes
two branches to extract different types of knowledge. The Weighted Concept
Knowledge (WCK) branch is responsible for introducing clinical medical concepts
weighted by TF-IDF scores. The Multimodal Retrieval Knowledge (MRK) branch
extracts triplets from similar reports, emphasizing crucial clinical
information related to entity positions and existence. By integrating this
finer-grained and well-structured knowledge with the current image, we are able
to leverage the multi-source knowledge gain to ultimately facilitate more
accurate report generation. Extensive experiments have been conducted on two
public benchmarks, demonstrating that our method achieves superior performance
over other state-of-the-art methods. Ablation studies further validate the
effectiveness of two extracted knowledge sources.