RelVAE: Generative Pretraining for few-shot Visual Relationship
  Detection
Sotiris Karapiperis Markos Diomataris Vassilis Pitsikalis
Visual relations are complex, multimodal concepts that play an important role
in the way humans perceive the world. As a result of their complexity,
high-quality, diverse and large scale datasets for visual relations are still
absent. In an attempt to overcome this data barrier, we choose to focus on the
problem of few-shot Visual Relationship Detection (VRD), a setting that has
been so far neglected by the community. In this work we present the first
pretraining method for few-shot predicate classification that does not require
any annotated relations. We achieve this by introducing a generative model that
is able to capture the variation of semantic, visual and spatial information of
relations inside a latent space and later exploiting its representations in
order to achieve efficient few-shot classification. We construct few-shot
training splits and show quantitative experiments on VG200 and VRD datasets
where our model outperforms the baselines. Lastly we attempt to interpret the
decisions of the model by conducting various qualitative experiments.