Does Graph Distillation See Like Vision Dataset Counterpart?
Beining Yang Kai Wang Qingyun Sun Cheng Ji Xingcheng Fu Hao Tang Yang You Jianxin Li
Training on large-scale graphs has achieved remarkable results in graph
representation learning, but its cost and storage have attracted increasing
concerns. Existing graph condensation methods primarily focus on optimizing the
feature matrices of condensed graphs while overlooking the impact of the
structure information from the original graphs. To investigate the impact of
the structure information, we conduct analysis from the spectral domain and
empirically identify substantial Laplacian Energy Distribution (LED) shifts in
previous works. Such shifts lead to poor performance in cross-architecture
generalization and specific tasks, including anomaly detection and link
prediction. In this paper, we propose a novel Structure-broadcasting Graph
Dataset Distillation (SGDD) scheme for broadcasting the original structure
information to the generation of the synthetic one, which explicitly prevents
overlooking the original structure information. Theoretically, the synthetic
graphs by SGDD are expected to have smaller LED shifts than previous works,
leading to superior performance in both cross-architecture settings and
specific tasks. We validate the proposed SGDD across 9 datasets and achieve
state-of-the-art results on all of them: for example, on the YelpChi dataset,
our approach maintains 98.6% test accuracy of training on the original graph
dataset with 1,000 times saving on the scale of the graph. Moreover, we
empirically evaluate there exist 17.6% ~ 31.4% reductions in LED shift crossing
9 datasets. Extensive experiments and analysis verify the effectiveness and
necessity of the proposed designs. The code is available in the GitHub
repository: https://github.com/RingBDStack/SGDD.