Pick the Best Pre-trained Model: Towards Transferability Estimation for
  Medical Image Segmentation
Yuncheng Yang Meng Wei Junjun He Jie Yang Jin Ye Yun Gu
Transfer learning is a critical technique in training deep neural networks
for the challenging medical image segmentation task that requires enormous
resources. With the abundance of medical image data, many research institutions
release models trained on various datasets that can form a huge pool of
candidate source models to choose from. Hence, it's vital to estimate the
source models' transferability (i.e., the ability to generalize across
different downstream tasks) for proper and efficient model reuse. To make up
for its deficiency when applying transfer learning to medical image
segmentation, in this paper, we therefore propose a new Transferability
Estimation (TE) method. We first analyze the drawbacks of using the existing TE
algorithms for medical image segmentation and then design a source-free TE
framework that considers both class consistency and feature variety for better
estimation. Extensive experiments show that our method surpasses all current
algorithms for transferability estimation in medical image segmentation. Code
is available at https://github.com/EndoluminalSurgicalVision-IMR/CCFV