MuseGNN: Interpretable and Convergent Graph Neural Network Layers at
  Scale
Haitian Jiang Renjie Liu Xiao Yan Zhenkun Cai Minjie Wang David Wipf
Among the many variants of graph neural network (GNN) architectures capable
of modeling data with cross-instance relations, an important subclass involves
layers designed such that the forward pass iteratively reduces a
graph-regularized energy function of interest. In this way, node embeddings
produced at the output layer dually serve as both predictive features for
solving downstream tasks (e.g., node classification) and energy function
minimizers that inherit desirable inductive biases and interpretability.
However, scaling GNN architectures constructed in this way remains challenging,
in part because the convergence of the forward pass may involve models with
considerable depth. To tackle this limitation, we propose a sampling-based
energy function and scalable GNN layers that iteratively reduce it, guided by
convergence guarantees in certain settings. We also instantiate a full GNN
architecture based on these designs, and the model achieves competitive
accuracy and scalability when applied to the largest publicly-available node
classification benchmark exceeding 1TB in size.