ExGRG: Explicitly-Generated Relation Graph for Self-Supervised
  Representation Learning
Mahdi Naseri Mahdi Biparva
Self-supervised Learning (SSL) has emerged as a powerful technique in
pre-training deep learning models without relying on expensive annotated
labels, instead leveraging embedded signals in unlabeled data. While SSL has
shown remarkable success in computer vision tasks through intuitive data
augmentation, its application to graph-structured data poses challenges due to
the semantic-altering and counter-intuitive nature of graph augmentations.
Addressing this limitation, this paper introduces a novel non-contrastive SSL
approach to Explicitly Generate a compositional Relation Graph (ExGRG) instead
of relying solely on the conventional augmentation-based implicit relation
graph. ExGRG offers a framework for incorporating prior domain knowledge and
online extracted information into the SSL invariance objective, drawing
inspiration from the Laplacian Eigenmap and Expectation-Maximization (EM).
Employing an EM perspective on SSL, our E-step involves relation graph
generation to identify candidates to guide the SSL invariance objective, and
M-step updates the model parameters by integrating the derived relational
information. Extensive experimentation on diverse node classification datasets
demonstrates the superiority of our method over state-of-the-art techniques,
affirming ExGRG as an effective adoption of SSL for graph representation
learning.