A Multimodal Approach for Cross-Domain Image Retrieval
Lucas Iijima Tania Stathaki
Image generators are gaining vast amount of popularity and have rapidly
changed how digital content is created. With the latest AI technology, millions
of high quality images are being generated by the public, which are constantly
motivating the research community to push the limits of generative models to
create more complex and realistic images. This paper focuses on Cross-Domain
Image Retrieval (CDIR) which can be used as an additional tool to inspect
collections of generated images by determining the level of similarity between
images in a dataset. An ideal retrieval system would be able to generalize to
unseen complex images from multiple domains (e.g., photos, drawings and
paintings). To address this goal, we propose a novel caption-matching approach
that leverages multimodal language-vision architectures pre-trained on large
datasets. The method is tested on DomainNet and Office-Home datasets and
consistently achieves state-of-the-art performance over the latest approaches
in the literature for cross-domain image retrieval. In order to verify the
effectiveness with AI-generated images, the method was also put to test with a
database composed by samples collected from Midjourney, which is a widely used
generative platform for content creation.