Benchmarking Sensitivity of Continual Graph Learning for Skeleton-Based
  Action Recognition
Wei Wei Tom De Schepper Kevin Mets
Continual learning (CL) is the research field that aims to build machine
learning models that can accumulate knowledge continuously over different tasks
without retraining from scratch. Previous studies have shown that pre-training
graph neural networks (GNN) may lead to negative transfer (Hu et al., 2020)
after fine-tuning, a setting which is closely related to CL. Thus, we focus on
studying GNN in the continual graph learning (CGL) setting. We propose the
first continual graph learning benchmark for spatio-temporal graphs and use it
to benchmark well-known CGL methods in this novel setting. The benchmark is
based on the N-UCLA and NTU-RGB+D datasets for skeleton-based action
recognition. Beyond benchmarking for standard performance metrics, we study the
class and task-order sensitivity of CGL methods, i.e., the impact of learning
order on each class/task's performance, and the architectural sensitivity of
CGL methods with backbone GNN at various widths and depths. We reveal that
task-order robust methods can still be class-order sensitive and observe
results that contradict previous empirical observations on architectural
sensitivity in CL.