Evolving Semantic Prototype Improves Generative Zero-Shot Learning
Shiming Chen Wenjin Hou Ziming Hong Xiaohan Ding Yibing Song Xinge You Tongliang Liu Kun Zhang
In zero-shot learning (ZSL), generative methods synthesize class-related
sample features based on predefined semantic prototypes. They advance the ZSL
performance by synthesizing unseen class sample features for better training
the classifier. We observe that each class's predefined semantic prototype
(also referred to as semantic embedding or condition) does not accurately match
its real semantic prototype. So the synthesized visual sample features do not
faithfully represent the real sample features, limiting the classifier training
and existing ZSL performance. In this paper, we formulate this mismatch
phenomenon as the visual-semantic domain shift problem. We propose a dynamic
semantic prototype evolving (DSP) method to align the empirically predefined
semantic prototypes and the real prototypes for class-related feature
synthesis. The alignment is learned by refining sample features and semantic
prototypes in a unified framework and making the synthesized visual sample
features approach real sample features. After alignment, synthesized sample
features from unseen classes are closer to the real sample features and benefit
DSP to improve existing generative ZSL methods by 8.5\%, 8.0\%, and 9.7\% on
the standard CUB, SUN AWA2 datasets, the significant performance improvement
indicates that evolving semantic prototype explores a virgin field in ZSL.