Vision Relation Transformer for Unbiased Scene Graph Generation
Gopika Sudhakaran Devendra Singh Dhami Kristian Kersting Stefan Roth
Recent years have seen a growing interest in Scene Graph Generation (SGG), a
comprehensive visual scene understanding task that aims to predict entity
relationships using a relation encoder-decoder pipeline stacked on top of an
object encoder-decoder backbone. Unfortunately, current SGG methods suffer from
an information loss regarding the entities local-level cues during the relation
encoding process. To mitigate this, we introduce the Vision rElation
TransfOrmer (VETO), consisting of a novel local-level entity relation encoder.
We further observe that many existing SGG methods claim to be unbiased, but are
still biased towards either head or tail classes. To overcome this bias, we
introduce a Mutually Exclusive ExperT (MEET) learning strategy that captures
important relation features without bias towards head or tail classes.
Experimental results on the VG and GQA datasets demonstrate that VETO + MEET
boosts the predictive performance by up to 47 percentage over the state of the
art while being 10 times smaller.