PUMA: Efficient Continual Graph Learning with Graph Condensation
Yilun Liu Ruihong Qiu Yanran Tang Hongzhi Yin Zi Huang
When handling streaming graphs, existing graph representation learning models
encounter a catastrophic forgetting problem, where previously learned knowledge
of these models is easily overwritten when learning with newly incoming graphs.
In response, Continual Graph Learning emerges as a novel paradigm enabling
graph representation learning from static to streaming graphs. Our prior work,
CaT is a replay-based framework with a balanced continual learning procedure,
which designs a small yet effective memory bank for replaying data by
condensing incoming graphs. Although the CaT alleviates the catastrophic
forgetting problem, there exist three issues: (1) The graph condensation
algorithm derived in CaT only focuses on labelled nodes while neglecting
abundant information carried by unlabelled nodes; (2) The continual training
scheme of the CaT overemphasises on the previously learned knowledge, limiting
the model capacity to learn from newly added memories; (3) Both the
condensation process and replaying process of the CaT are time-consuming. In
this paper, we propose a psudo-label guided memory bank (PUMA) CGL framework,
extending from the CaT to enhance its efficiency and effectiveness by
overcoming the above-mentioned weaknesses and limits. To fully exploit the
information in a graph, PUMA expands the coverage of nodes during graph
condensation with both labelled and unlabelled nodes. Furthermore, a
training-from-scratch strategy is proposed to upgrade the previous continual
learning scheme for a balanced training between the historical and the new
graphs. Besides, PUMA uses a one-time prorogation and wide graph encoders to
accelerate the graph condensation and the graph encoding process in the
training stage to improve the efficiency of the whole framework. Extensive
experiments on four datasets demonstrate the state-of-the-art performance and
efficiency over existing methods.