From Region to Patch: Attribute-Aware Foreground-Background Contrastive
  Learning for Fine-Grained Fashion Retrieval
Jianfeng Dong Xiaoman Peng Zhe Ma Daizong Liu Xiaoye Qu Xun Yang Jixiang Zhu Baolong Liu
Attribute-specific fashion retrieval (ASFR) is a challenging information
retrieval task, which has attracted increasing attention in recent years.
Different from traditional fashion retrieval which mainly focuses on optimizing
holistic similarity, the ASFR task concentrates on attribute-specific
similarity, resulting in more fine-grained and interpretable retrieval results.
As the attribute-specific similarity typically corresponds to the specific
subtle regions of images, we propose a Region-to-Patch Framework (RPF) that
consists of a region-aware branch and a patch-aware branch to extract
fine-grained attribute-related visual features for precise retrieval in a
coarse-to-fine manner. In particular, the region-aware branch is first to be
utilized to locate the potential regions related to the semantic of the given
attribute. Then, considering that the located region is coarse and still
contains the background visual contents, the patch-aware branch is proposed to
capture patch-wise attribute-related details from the previous amplified
region. Such a hybrid architecture strikes a proper balance between region
localization and feature extraction. Besides, different from previous works
that solely focus on discriminating the attribute-relevant foreground visual
features, we argue that the attribute-irrelevant background features are also
crucial for distinguishing the detailed visual contexts in a contrastive
manner. Therefore, a novel E-InfoNCE loss based on the foreground and
background representations is further proposed to improve the discrimination of
attribute-specific representation. Extensive experiments on three datasets
demonstrate the effectiveness of our proposed framework, and also show a decent
generalization of our RPF on out-of-domain fashion images. Our source code is
available at https://github.com/HuiGuanLab/RPF.