Text as Image: Learning Transferable Adapter for Multi-Label
  Classification
Xuelin Zhu Jiuxin Cao Jian liu Dongqi Tang Furong Xu Weijia Liu Jiawei Ge Bo Liu Qingpei Guo Tianyi Zhang
Pre-trained vision-language models have notably accelerated progress of
open-world concept recognition. Their impressive zero-shot ability has recently
been transferred to multi-label image classification via prompt tuning,
enabling to discover novel labels in an open-vocabulary manner. However, this
paradigm suffers from non-trivial training costs, and becomes computationally
prohibitive for a large number of candidate labels. To address this issue, we
note that vision-language pre-training aligns images and texts in a unified
embedding space, making it potential for an adapter network to identify labels
in visual modality while be trained in text modality. To enhance such
cross-modal transfer ability, a simple yet effective method termed random
perturbation is proposed, which enables the adapter to search for potential
visual embeddings by perturbing text embeddings with noise during training,
resulting in better performance in visual modality. Furthermore, we introduce
an effective approach to employ large language models for multi-label
instruction-following text generation. In this way, a fully automated pipeline
for visual label recognition is developed without relying on any manual data.
Extensive experiments on public benchmarks show the superiority of our method
in various multi-label classification tasks.