General Purpose Image Encoder DINOv2 for Medical Image Registration
Xinrui Song Xuanang Xu Pingkun Yan
Existing medical image registration algorithms rely on either dataset
specific training or local texture-based features to align images. The former
cannot be reliably implemented without large modality-specific training
datasets, while the latter lacks global semantics thus could be easily trapped
at local minima. In this paper, we present a training-free deformable image
registration method, DINO-Reg, leveraging a general purpose image encoder
DINOv2 for image feature extraction. The DINOv2 encoder was trained using the
ImageNet data containing natural images. We used the pretrained DINOv2 without
any finetuning. Our method feeds the DINOv2 encoded features into a discrete
optimizer to find the optimal deformable registration field. We conducted a
series of experiments to understand the behavior and role of such a general
purpose image encoder in the application of image registration. Combined with
handcrafted features, our method won the first place in the recent OncoReg
Challenge. To our knowledge, this is the first application of general vision
foundation models in medical image registration.