Graph Neural Aggregation-diffusion with Metastability
Kaiyuan Cui Xinyan Wang Zicheng Zhang Weichen Zhao
Continuous graph neural models based on differential equations have expanded
the architecture of graph neural networks (GNNs). Due to the connection between
graph diffusion and message passing, diffusion-based models have been widely
studied. However, diffusion naturally drives the system towards an equilibrium
state, leading to issues like over-smoothing. To this end, we propose GRADE
inspired by graph aggregation-diffusion equations, which includes the delicate
balance between nonlinear diffusion and aggregation induced by interaction
potentials. The node representations obtained through aggregation-diffusion
equations exhibit metastability, indicating that features can aggregate into
multiple clusters. In addition, the dynamics within these clusters can persist
for long time periods, offering the potential to alleviate over-smoothing
effects. This nonlinear diffusion in our model generalizes existing
diffusion-based models and establishes a connection with classical GNNs. We
prove that GRADE achieves competitive performance across various benchmarks and
alleviates the over-smoothing issue in GNNs evidenced by the enhanced Dirichlet
energy.