MatchSeg: Towards Better Segmentation via Reference Image Matching
Ruiqiang Xiao Jiayu Huo Haotian Zheng Yang Liu Sebastien Ourselin Rachel Sparks
Recently, automated medical image segmentation methods based on deep learning
have achieved great success. However, they heavily rely on large annotated
datasets, which are costly and time-consuming to acquire. Few-shot learning
aims to overcome the need for annotated data by using a small labeled dataset,
known as a support set, to guide predicting labels for new, unlabeled images,
known as the query set. Inspired by this paradigm, we introduce MatchSeg, a
novel framework that enhances medical image segmentation through strategic
reference image matching. We leverage contrastive language-image pre-training
(CLIP) to select highly relevant samples when defining the support set.
Additionally, we design a joint attention module to strengthen the interaction
between support and query features, facilitating a more effective knowledge
transfer between support and query sets. We validated our method across four
public datasets. Experimental results demonstrate superior segmentation
performance and powerful domain generalization ability of MatchSeg against
existing methods for domain-specific and cross-domain segmentation tasks. Our
code is made available at https://github.com/keeplearning-again/MatchSeg