Zero-Shot Learning by Harnessing Adversarial Samples
Zhi Chen Pengfei Zhang Jingjing Li Sen Wang Zi Huang
Zero-Shot Learning (ZSL) aims to recognize unseen classes by generalizing the
knowledge, i.e., visual and semantic relationships, obtained from seen classes,
where image augmentation techniques are commonly applied to improve the
generalization ability of a model. However, this approach can also cause
adverse effects on ZSL since the conventional augmentation techniques that
solely depend on single-label supervision is not able to maintain semantic
information and result in the semantic distortion issue consequently. In other
words, image argumentation may falsify the semantic (e.g., attribute)
information of an image. To take the advantage of image augmentations while
mitigating the semantic distortion issue, we propose a novel ZSL approach by
Harnessing Adversarial Samples (HAS). HAS advances ZSL through adversarial
training which takes into account three crucial aspects: (1) robust generation
by enforcing augmentations to be similar to negative classes, while maintaining
correct labels, (2) reliable generation by introducing a latent space
constraint to avert significant deviations from the original data manifold, and
(3) diverse generation by incorporating attribute-based perturbation by
adjusting images according to each semantic attribute's localization. Through
comprehensive experiments on three prominent zero-shot benchmark datasets, we
demonstrate the effectiveness of our adversarial samples approach in both ZSL
and Generalized Zero-Shot Learning (GZSL) scenarios. Our source code is
available at https://github.com/uqzhichen/HASZSL.