Generative and Contrastive Paradigms Are Complementary for Graph
  Self-Supervised Learning
Yuxiang Wang Xiao Yan Chuang Hu Fangcheng Fu Wentao Zhang Hao Wang Shuo Shang Jiawei Jiang
For graph self-supervised learning (GSSL), masked autoencoder (MAE) follows
the generative paradigm and learns to reconstruct masked graph edges or node
features. Contrastive Learning (CL) maximizes the similarity between augmented
views of the same graph and is widely used for GSSL. However, MAE and CL are
considered separately in existing works for GSSL. We observe that the MAE and
CL paradigms are complementary and propose the graph contrastive masked
autoencoder (GCMAE) framework to unify them. Specifically, by focusing on local
edges or node features, MAE cannot capture global information of the graph and
is sensitive to particular edges and features. On the contrary, CL excels in
extracting global information because it considers the relation between graphs.
As such, we equip GCMAE with an MAE branch and a CL branch, and the two
branches share a common encoder, which allows the MAE branch to exploit the
global information extracted by the CL branch. To force GCMAE to capture global
graph structures, we train it to reconstruct the entire adjacency matrix
instead of only the masked edges as in existing works. Moreover, a
discrimination loss is proposed for feature reconstruction, which improves the
disparity between node embeddings rather than reducing the reconstruction error
to tackle the feature smoothing problem of MAE. We evaluate GCMAE on four
popular graph tasks (i.e., node classification, node clustering, link
prediction, and graph classification) and compare with 14 state-of-the-art
baselines. The results show that GCMAE consistently provides good accuracy
across these tasks, and the maximum accuracy improvement is up to 3.2% compared
with the best-performing baseline.