A Contrastive Variational Graph Auto-Encoder for Node Clustering
Nairouz Mrabah Mohamed Bouguessa Riadh Ksantini
Variational Graph Auto-Encoders (VGAEs) have been widely used to solve the
node clustering task. However, the state-of-the-art methods have numerous
challenges. First, existing VGAEs do not account for the discrepancy between
the inference and generative models after incorporating the clustering
inductive bias. Second, current models are prone to degenerate solutions that
make the latent codes match the prior independently of the input signal (i.e.,
Posterior Collapse). Third, existing VGAEs overlook the effect of the noisy
clustering assignments (i.e., Feature Randomness) and the impact of the strong
trade-off between clustering and reconstruction (i.e., Feature Drift). To
address these problems, we formulate a variational lower bound in a contrastive
setting. Our lower bound is a tighter approximation of the log-likelihood
function than the corresponding Evidence Lower BOund (ELBO). Thanks to a newly
identified term, our lower bound can escape Posterior Collapse and has more
flexibility to account for the difference between the inference and generative
models. Additionally, our solution has two mechanisms to control the trade-off
between Feature Randomness and Feature Drift. Extensive experiments show that
the proposed method achieves state-of-the-art clustering results on several
datasets. We provide strong evidence that this improvement is attributed to
four aspects: integrating contrastive learning and alleviating Feature
Randomness, Feature Drift, and Posterior Collapse.