Entropy Neural Estimation for Graph Contrastive Learning
Yixuan Ma Xiaolin Zhang Peng Zhang Kun Zhan
Contrastive learning on graphs aims at extracting distinguishable high-level
representations of nodes. In this paper, we theoretically illustrate that the
entropy of a dataset can be approximated by maximizing the lower bound of the
mutual information across different views of a graph, \ie, entropy is estimated
by a neural network. Based on this finding, we propose a simple yet effective
subset sampling strategy to contrast pairwise representations between views of
a dataset. In particular, we randomly sample nodes and edges from a given graph
to build the input subset for a view. Two views are fed into a parameter-shared
Siamese network to extract the high-dimensional embeddings and estimate the
information entropy of the entire graph. For the learning process, we propose
to optimize the network using two objectives, simultaneously. Concretely, the
input of the contrastive loss function consists of positive and negative pairs.
Our selection strategy of pairs is different from previous works and we present
a novel strategy to enhance the representation ability of the graph encoder by
selecting nodes based on cross-view similarities. We enrich the diversity of
the positive and negative pairs by selecting highly similar samples and totally
different data with the guidance of cross-view similarity scores, respectively.
We also introduce a cross-view consistency constraint on the representations
generated from the different views. This objective guarantees the learned
representations are consistent across views from the perspective of the entire
graph. We conduct extensive experiments on seven graph benchmarks, and the
proposed approach achieves competitive performance compared to the current
state-of-the-art methods. The source code will be publicly released once this
paper is accepted.