Independent Distribution Regularization for Private Graph Embedding
Qi Hu Yangqiu Song
Learning graph embeddings is a crucial task in graph mining tasks. An
effective graph embedding model can learn low-dimensional representations from
graph-structured data for data publishing benefiting various downstream
applications such as node classification, link prediction, etc. However, recent
studies have revealed that graph embeddings are susceptible to attribute
inference attacks, which allow attackers to infer private node attributes from
the learned graph embeddings. To address these concerns, privacy-preserving
graph embedding methods have emerged, aiming to simultaneously consider primary
learning and privacy protection through adversarial learning. However, most
existing methods assume that representation models have access to all sensitive
attributes in advance during the training stage, which is not always the case
due to diverse privacy preferences. Furthermore, the commonly used adversarial
learning technique in privacy-preserving representation learning suffers from
unstable training issues. In this paper, we propose a novel approach called
Private Variational Graph AutoEncoders (PVGAE) with the aid of independent
distribution penalty as a regularization term. Specifically, we split the
original variational graph autoencoder (VGAE) to learn sensitive and
non-sensitive latent representations using two sets of encoders. Additionally,
we introduce a novel regularization to enforce the independence of the
encoders. We prove the theoretical effectiveness of regularization from the
perspective of mutual information. Experimental results on three real-world
datasets demonstrate that PVGAE outperforms other baselines in private
embedding learning regarding utility performance and privacy protection.