UGMAE: A Unified Framework for Graph Masked Autoencoders
Yijun Tian Chuxu Zhang Ziyi Kou Zheyuan Liu Xiangliang Zhang Nitesh V. Chawla
Generative self-supervised learning on graphs, particularly graph masked
autoencoders, has emerged as a popular learning paradigm and demonstrated its
efficacy in handling non-Euclidean data. However, several remaining issues
limit the capability of existing methods: 1) the disregard of uneven node
significance in masking, 2) the underutilization of holistic graph information,
3) the ignorance of semantic knowledge in the representation space due to the
exclusive use of reconstruction loss in the output space, and 4) the unstable
reconstructions caused by the large volume of masked contents. In light of
this, we propose UGMAE, a unified framework for graph masked autoencoders to
address these issues from the perspectives of adaptivity, integrity,
complementarity, and consistency. Specifically, we first develop an adaptive
feature mask generator to account for the unique significance of nodes and
sample informative masks (adaptivity). We then design a ranking-based structure
reconstruction objective joint with feature reconstruction to capture holistic
graph information and emphasize the topological proximity between neighbors
(integrity). After that, we present a bootstrapping-based similarity module to
encode the high-level semantic knowledge in the representation space,
complementary to the low-level reconstruction in the output space
(complementarity). Finally, we build a consistency assurance module to provide
reconstruction objectives with extra stabilized consistency targets
(consistency). Extensive experiments demonstrate that UGMAE outperforms both
contrastive and generative state-of-the-art baselines on several tasks across
multiple datasets.