A foundation model utilizing chest CT volumes and radiology reports for
  supervised-level zero-shot detection of abnormalities
Ibrahim Ethem Hamamci Sezgin Er Furkan Almas Ayse Gulnihan Simsek Sevval Nil Esirgun Irem Dogan Muhammed Furkan Dasdelen Bastian Wittmann Enis Simsar Mehmet Simsar Emine Bensu Erdemir Abdullah Alanbay Anjany Sekuboyina Berkan Lafci Mehmet K. Ozdemir Bjoern Menze
A major challenge in computational research in 3D medical imaging is the lack
of comprehensive datasets. Addressing this issue, our study introduces CT-RATE,
the first 3D medical imaging dataset that pairs images with textual reports.
CT-RATE consists of 25,692 non-contrast chest CT volumes, expanded to 50,188
through various reconstructions, from 21,304 unique patients, along with
corresponding radiology text reports. Leveraging CT-RATE, we developed CT-CLIP,
a CT-focused contrastive language-image pre-training framework. As a versatile,
self-supervised model, CT-CLIP is designed for broad application and does not
require task-specific training. Remarkably, CT-CLIP outperforms
state-of-the-art, fully supervised methods in multi-abnormality detection
across all key metrics, thus eliminating the need for manual annotation. We
also demonstrate its utility in case retrieval, whether using imagery or
textual queries, thereby advancing knowledge dissemination. The open-source
release of CT-RATE and CT-CLIP marks a significant advancement in medical AI,
enhancing 3D imaging analysis and fostering innovation in healthcare.