VISION-MAE: A Foundation Model for Medical Image Segmentation and
  Classification
Zelong Liu Andrew Tieu Nikhil Patel Alexander Zhou George Soultanidis Zahi A. Fayad Timothy Deyer Xueyan Mei
Artificial Intelligence (AI) has the potential to revolutionize diagnosis and
segmentation in medical imaging. However, development and clinical
implementation face multiple challenges including limited data availability,
lack of generalizability, and the necessity to incorporate multi-modal data
effectively. A foundation model, which is a large-scale pre-trained AI model,
offers a versatile base that can be adapted to a variety of specific tasks and
contexts. Here, we present a novel foundation model, VISION-MAE, specifically
designed for medical imaging. Specifically, VISION-MAE is trained on a dataset
of 2.5 million unlabeled images from various modalities (CT, MR, PET, X-rays,
and ultrasound), using self-supervised learning techniques. It is then adapted
to classification and segmentation tasks using explicit labels. VISION-MAE has
high label efficiency, outperforming several benchmark models in both in-domain
and out-of-domain applications, and achieves high performance even with reduced
availability of labeled data. This model represents a significant advancement
in medical imaging AI, offering a generalizable and robust solution for
improving segmentation and classification tasks while reducing the data
annotation workload.