RadCLIP: Enhancing Radiologic Image Analysis through Contrastive
  Language-Image Pre-training
Zhixiu Lu Hailong Li Lili He
The integration of artificial intelligence (AI) with radiology has marked a
transformative era in medical diagnostics. Vision foundation models have been
adopted to enhance radiologic imaging analysis. However, the distinct
complexities of radiological imaging, including the interpretation of 2D and 3D
radiological data, pose unique challenges that existing models, trained on
general non-medical images, fail to address adequately. To bridge this gap and
capitalize on the diagnostic precision required in medical imaging, we
introduce RadCLIP: a pioneering cross-modal foundational model that harnesses
Contrastive Language-Image Pre-training (CLIP) to refine radiologic image
analysis. RadCLIP incorporates a novel 3D slice pooling mechanism tailored for
volumetric image analysis and is trained using a comprehensive and diverse
dataset of radiologic image-text pairs. Our evaluations demonstrate that
RadCLIP effectively aligns radiological images with their corresponding textual
annotations, and in the meantime, offers a robust vision backbone for
radiologic imagery with significant promise.