Self-Enhancement Improves Text-Image Retrieval in Foundation
  Visual-Language Models
Yuguang Yang Yiming Wang Shupeng Geng Runqi Wang Yimi Wang Sheng Wu Baochang Zhang
The emergence of cross-modal foundation models has introduced numerous
approaches grounded in text-image retrieval. However, on some domain-specific
retrieval tasks, these models fail to focus on the key attributes required. To
address this issue, we propose a self-enhancement framework, A^{3}R, based on
the CLIP-ViT/G-14, one of the largest cross-modal models. First, we perform an
Attribute Augmentation strategy to enrich the textual description for
fine-grained representation before model learning. Then, we propose an Adaption
Re-ranking method to unify the representation space of textual query and
candidate images and re-rank candidate images relying on the adapted query
after model learning. The proposed framework is validated to achieve a salient
improvement over the baseline and other teams' solutions in the cross-modal
image retrieval track of the 1st foundation model challenge without introducing
any additional samples. The code is available at
\url{https://github.com/CapricornGuang/A3R}.