Predicate Classification Using Optimal Transport Loss in Scene Graph
  Generation
Sorachi Kurita Satoshi Oyama Itsuki Noda
In scene graph generation (SGG), learning with cross-entropy loss yields
biased predictions owing to the severe imbalance in the distribution of the
relationship labels in the dataset. Thus, this study proposes a method to
generate scene graphs using optimal transport as a measure for comparing two
probability distributions. We apply learning with the optimal transport loss,
which reflects the similarity between the labels in terms of transportation
cost, for predicate classification in SGG. In the proposed approach, the
transportation cost of the optimal transport is defined using the similarity of
words obtained from the pre-trained model. The experimental evaluation of the
effectiveness demonstrates that the proposed method outperforms existing
methods in terms of mean Recall@50 and 100. Furthermore, it improves the recall
of the relationship labels scarcely available in the dataset.