Complexity-Guided Curriculum Learning for Text Graphs
Nidhi Vakil Hadi Amiri
Curriculum learning provides a systematic approach to training. It refines
training progressively, tailors training to task requirements, and improves
generalization through exposure to diverse examples. We present a curriculum
learning approach that builds on existing knowledge about text and graph
complexity formalisms for training with text graph data. The core part of our
approach is a novel data scheduler, which employs "spaced repetition" and
complexity formalisms to guide the training process. We demonstrate the
effectiveness of the proposed approach on several text graph tasks and graph
neural network architectures. The proposed model gains more and uses less data;
consistently prefers text over graph complexity indices throughout training,
while the best curricula derived from text and graph complexity indices are
equally effective; and it learns transferable curricula across GNN models and
datasets. In addition, we find that both node-level (local) and graph-level
(global) graph complexity indices, as well as shallow and traditional text
complexity indices play a crucial role in effective curriculum learning.