Overcoming Data Scarcity in Biomedical Imaging with a Foundational
  Multi-Task Model
Raphael Schäfer Till Nicke Henning Höfener Annkristin Lange Dorit Merhof Friedrich Feuerhake Volkmar Schulz Johannes Lotz Fabian Kiessling
Foundational models, pretrained on a large scale, have demonstrated
substantial success across non-medical domains. However, training these models
typically requires large, comprehensive datasets, which contrasts with the
smaller and more heterogeneous datasets common in biomedical imaging. Here, we
propose a multi-task learning strategy that decouples the number of training
tasks from memory requirements. We trained a Universal bioMedical PreTrained
model (UMedPT) on a multi-task database including tomographic, microscopic, and
X-ray images, with various labelling strategies such as classification,
segmentation, and object detection. The UMedPT foundational model outperformed
ImageNet pretraining and the previous state-of-the-art models. For tasks
related to the pretraining database, it maintained its performance with only 1%
of the original training data and without fine-tuning. For out-of-domain tasks
it required not more than 50% of the original training data. In an external
independent validation imaging features extracted using UMedPT proved to be a
new standard for cross-center transferability.