Semantic Scene Graph Generation Based on an Edge Dual Scene Graph and
  Message Passing Neural Network
Hyeongjin Kim Sangwon Kim Jong Taek Lee Byoung Chul Ko
Along with generative AI, interest in scene graph generation (SGG), which
comprehensively captures the relationships and interactions between objects in
an image and creates a structured graph-based representation, has significantly
increased in recent years. However, relying on object-centric and dichotomous
relationships, existing SGG methods have a limited ability to accurately
predict detailed relationships. To solve these problems, a new approach to the
modeling multiobject relationships, called edge dual scene graph generation
(EdgeSGG), is proposed herein. EdgeSGG is based on a edge dual scene graph and
Dual Message Passing Neural Network (DualMPNN), which can capture rich
contextual interactions between unconstrained objects. To facilitate the
learning of edge dual scene graphs with a symmetric graph structure, the
proposed DualMPNN learns both object- and relation-centric features for more
accurately predicting relation-aware contexts and allows fine-grained
relational updates between objects. A comparative experiment with
state-of-the-art (SoTA) methods was conducted using two public datasets for SGG
operations and six metrics for three subtasks. Compared with SoTA approaches,
the proposed model exhibited substantial performance improvements across all
SGG subtasks. Furthermore, experiment on long-tail distributions revealed that
incorporating the relationships between objects effectively mitigates existing
long-tail problems.