Generalized Unbiased Scene Graph Generation
Xinyu Lyu Lianli Gao Junlin Xie Pengpeng Zeng Yulu Tian Jie Shao Heng Tao Shen
Existing Unbiased Scene Graph Generation (USGG) methods only focus on
addressing the predicate-level imbalance that high-frequency classes dominate
predictions of rare ones, while overlooking the concept-level imbalance.
Actually, even if predicates themselves are balanced, there is still a
significant concept-imbalance within them due to the long-tailed distribution
of contexts (i.e., subject-object combinations). This concept-level imbalance
poses a more pervasive and challenging issue compared to the predicate-level
imbalance since subject-object pairs are inherently complex in combinations.
Hence, we introduce a novel research problem: Generalized Unbiased Scene Graph
Generation (G-USGG), which takes into account both predicate-level and
concept-level imbalance. To the end, we propose the Multi-Concept Learning
(MCL) framework, which ensures a balanced learning process across rare/
uncommon/ common concepts. MCL first quantifies the concept-level imbalance
across predicates in terms of different amounts of concepts, representing as
multiple concept-prototypes within the same class. It then effectively learns
concept-prototypes by applying the Concept Regularization (CR) technique.
Furthermore, to achieve balanced learning over different concepts, we introduce
the Balanced Prototypical Memory (BPM), which guides SGG models to generate
balanced representations for concept-prototypes. Extensive experiments
demonstrate the remarkable efficacy of our model-agnostic strategy in enhancing
the performance of benchmark models on both VG-SGG and OI-SGG datasets, leading
to new state-of-the-art achievements in two key aspects: predicate-level
unbiased relation recognition and concept-level compositional generability.