Prototypical Contrastive Learning-based CLIP Fine-tuning for Object
  Re-identification
Jiachen Li Xiaojin Gong
This work aims to adapt large-scale pre-trained vision-language models, such
as contrastive language-image pretraining (CLIP), to enhance the performance of
object reidentification (Re-ID) across various supervision settings. Although
prompt learning has enabled a recent work named CLIP-ReID to achieve promising
performance, the underlying mechanisms and the necessity of prompt learning
remain unclear due to the absence of semantic labels in ReID tasks. In this
work, we first analyze the role prompt learning in CLIP-ReID and identify its
limitations. Based on our investigations, we propose a simple yet effective
approach to adapt CLIP for supervised object Re-ID. Our approach directly
fine-tunes the image encoder of CLIP using a prototypical contrastive learning
(PCL) loss, eliminating the need for prompt learning. Experimental results on
both person and vehicle Re-ID datasets demonstrate the competitiveness of our
method compared to CLIP-ReID. Furthermore, we extend our PCL-based CLIP
fine-tuning approach to unsupervised scenarios, where we achieve state-of-the
art performance.