Image-free Classifier Injection for Zero-Shot Classification
Anders Christensen Massimiliano Mancini A. Sophia Koepke Ole Winther Zeynep Akata
Zero-shot learning models achieve remarkable results on image classification
for samples from classes that were not seen during training. However, such
models must be trained from scratch with specialised methods: therefore, access
to a training dataset is required when the need for zero-shot classification
arises. In this paper, we aim to equip pre-trained models with zero-shot
classification capabilities without the use of image data. We achieve this with
our proposed Image-free Classifier Injection with Semantics (ICIS) that injects
classifiers for new, unseen classes into pre-trained classification models in a
post-hoc fashion without relying on image data. Instead, the existing
classifier weights and simple class-wise descriptors, such as class names or
attributes, are used. ICIS has two encoder-decoder networks that learn to
reconstruct classifier weights from descriptors (and vice versa), exploiting
(cross-)reconstruction and cosine losses to regularise the decoding process.
Notably, ICIS can be cheaply trained and applied directly on top of pre-trained
classification models. Experiments on benchmark ZSL datasets show that ICIS
produces unseen classifier weights that achieve strong (generalised) zero-shot
classification performance. Code is available at
https://github.com/ExplainableML/ImageFreeZSL .