UniGraph: Learning a Cross-Domain Graph Foundation Model From Natural
  Language
Yufei He Bryan Hooi
Foundation models like ChatGPT and GPT-4 have revolutionized artificial
intelligence, exhibiting remarkable abilities to generalize across a wide array
of tasks and applications beyond their initial training objectives. However,
when this concept is applied to graph learning, a stark contrast emerges. Graph
learning has predominantly focused on single-graph models, tailored to specific
tasks or datasets, lacking the ability to transfer learned knowledge to
different domains. This limitation stems from the inherent complexity and
diversity of graph structures, along with the different feature and label
spaces specific to graph data. In this paper, we present our UniGraph
framework, designed to train a graph foundation model capable of generalizing
to unseen graphs and tasks across diverse domains. Unlike single-graph models
that use pre-computed node features of varying dimensions as input, our
approach leverages Text-Attributed Graphs (TAGs) for unifying node
representations. We propose a cascaded architecture of Language Models (LMs)
and Graph Neural Networks (GNNs) as backbone networks with a self-supervised
training objective based on Masked Graph Modeling (MGM). We introduce graph
instruction tuning using Large Language Models (LLMs) to enable zero-shot
prediction ability. Our comprehensive experiments across various graph learning
tasks and domains demonstrate the model's effectiveness in self-supervised
representation learning on unseen graphs, few-shot in-context transfer, and
zero-shot transfer, even surpassing or matching the performance of GNNs that
have undergone supervised training on target datasets.