Dual-View Data Hallucination with Semantic Relation Guidance for
  Few-Shot Image Recognition
Hefeng Wu Guangzhi Ye Ziyang Zhou Ling Tian Qing Wang Liang Lin
Learning to recognize novel concepts from just a few image samples is very
challenging as the learned model is easily overfitted on the few data and
results in poor generalizability. One promising but underexplored solution is
to compensate the novel classes by generating plausible samples. However, most
existing works of this line exploit visual information only, rendering the
generated data easy to be distracted by some challenging factors contained in
the few available samples. Being aware of the semantic information in the
textual modality that reflects human concepts, this work proposes a novel
framework that exploits semantic relations to guide dual-view data
hallucination for few-shot image recognition. The proposed framework enables
generating more diverse and reasonable data samples for novel classes through
effective information transfer from base classes. Specifically, an
instance-view data hallucination module hallucinates each sample of a novel
class to generate new data by employing local semantic correlated attention and
global semantic feature fusion derived from base classes. Meanwhile, a
prototype-view data hallucination module exploits semantic-aware measure to
estimate the prototype of a novel class and the associated distribution from
the few samples, which thereby harvests the prototype as a more stable sample
and enables resampling a large number of samples. We conduct extensive
experiments and comparisons with state-of-the-art methods on several popular
few-shot benchmarks to verify the effectiveness of the proposed framework.