A Metadata-Driven Approach to Understand Graph Neural Networks
Ting Wei Li Qiaozhu Mei Jiaqi Ma
Graph Neural Networks (GNNs) have achieved remarkable success in various
applications, but their performance can be sensitive to specific data
properties of the graph datasets they operate on. Current literature on
understanding the limitations of GNNs has primarily employed a
$\textit{model-driven}$ approach that leverage heuristics and domain knowledge
from network science or graph theory to model the GNN behaviors, which is
time-consuming and highly subjective. In this work, we propose a
$\textit{metadata-driven}$ approach to analyze the sensitivity of GNNs to graph
data properties, motivated by the increasing availability of graph learning
benchmarks. We perform a multivariate sparse regression analysis on the
metadata derived from benchmarking GNN performance across diverse datasets,
yielding a set of salient data properties. To validate the effectiveness of our
data-driven approach, we focus on one identified data property, the degree
distribution, and investigate how this property influences GNN performance
through theoretical analysis and controlled experiments. Our theoretical
findings reveal that datasets with more balanced degree distribution exhibit
better linear separability of node representations, thus leading to better GNN
performance. We also conduct controlled experiments using synthetic datasets
with varying degree distributions, and the results align well with our
theoretical findings. Collectively, both the theoretical analysis and
controlled experiments verify that the proposed metadata-driven approach is
effective in identifying critical data properties for GNNs.