Empower Text-Attributed Graphs Learning with Large Language Models
  (LLMs)
Jianxiang Yu Yuxiang Ren Chenghua Gong Jiaqi Tan Xiang Li Xuecang Zhang
Text-attributed graphs have recently garnered significant attention due to
their wide range of applications in web domains. Existing methodologies employ
word embedding models for acquiring text representations as node features,
which are subsequently fed into Graph Neural Networks (GNNs) for training.
Recently, the advent of Large Language Models (LLMs) has introduced their
powerful capabilities in information retrieval and text generation, which can
greatly enhance the text attributes of graph data. Furthermore, the acquisition
and labeling of extensive datasets are both costly and time-consuming
endeavors. Consequently, few-shot learning has emerged as a crucial problem in
the context of graph learning tasks. In order to tackle this challenge, we
propose a lightweight paradigm called ENG, which adopts a plug-and-play
approach to empower text-attributed graphs through node generation using LLMs.
Specifically, we utilize LLMs to extract semantic information from the labels
and generate samples that belong to these categories as exemplars.
Subsequently, we employ an edge predictor to capture the structural information
inherent in the raw dataset and integrate the newly generated samples into the
original graph. This approach harnesses LLMs for enhancing class-level
information and seamlessly introduces labeled nodes and edges without modifying
the raw dataset, thereby facilitating the node classification task in few-shot
scenarios. Extensive experiments demonstrate the outstanding performance of our
proposed paradigm, particularly in low-shot scenarios. For instance, in the
1-shot setting of the ogbn-arxiv dataset, ENG achieves a 76% improvement over
the baseline model.