Active Learning for Fine-Grained Sketch-Based Image Retrieval
Himanshu Thakur Soumitri Chattopadhyay
The ability to retrieve a photo by mere free-hand sketching highlights the
immense potential of Fine-grained sketch-based image retrieval (FG-SBIR).
However, its rapid practical adoption, as well as scalability, is limited by
the expense of acquiring faithful sketches for easily available photo
counterparts. A solution to this problem is Active Learning, which could
minimise the need for labeled sketches while maximising performance. Despite
extensive studies in the field, there exists no work that utilises it for
reducing sketching effort in FG-SBIR tasks. To this end, we propose a novel
active learning sampling technique that drastically minimises the need for
drawing photo sketches. Our proposed approach tackles the trade-off between
uncertainty and diversity by utilising the relationship between the existing
photo-sketch pair to a photo that does not have its sketch and augmenting this
relation with its intermediate representations. Since our approach relies only
on the underlying data distribution, it is agnostic of the modelling approach
and hence is applicable to other cross-modal instance-level retrieval tasks as
well. With experimentation over two publicly available fine-grained SBIR
datasets ChairV2 and ShoeV2, we validate our approach and reveal its
superiority over adapted baselines.