Re-Think and Re-Design Graph Neural Networks in Spaces of Continuous
  Graph Diffusion Functionals
Tingting Dan Jiaqi Ding Ziquan Wei Shahar Z Kovalsky Minjeong Kim Won Hwa Kim Guorong Wu
Graph neural networks (GNNs) are widely used in domains like social networks
and biological systems. However, the locality assumption of GNNs, which limits
information exchange to neighboring nodes, hampers their ability to capture
long-range dependencies and global patterns in graphs. To address this, we
propose a new inductive bias based on variational analysis, drawing inspiration
from the Brachistochrone problem. Our framework establishes a mapping between
discrete GNN models and continuous diffusion functionals. This enables the
design of application-specific objective functions in the continuous domain and
the construction of discrete deep models with mathematical guarantees. To
tackle over-smoothing in GNNs, we analyze the existing layer-by-layer graph
embedding models and identify that they are equivalent to l2-norm integral
functionals of graph gradients, which cause over-smoothing. Similar to
edge-preserving filters in image denoising, we introduce total variation (TV)
to align the graph diffusion pattern with global community topologies.
Additionally, we devise a selective mechanism to address the trade-off between
model depth and over-smoothing, which can be easily integrated into existing
GNNs. Furthermore, we propose a novel generative adversarial network (GAN) that
predicts spreading flows in graphs through a neural transport equation. To
mitigate vanishing flows, we customize the objective function to minimize
transportation within each community while maximizing inter-community flows.
Our GNN models achieve state-of-the-art (SOTA) performance on popular graph
learning benchmarks such as Cora, Citeseer, and Pubmed.