Decoupling Weighing and Selecting for Integrating Multiple Graph
  Pre-training Tasks
Tianyu Fan Lirong Wu Yufei Huang Haitao Lin Cheng Tan Zhangyang Gao Stan Z. Li
Recent years have witnessed the great success of graph pre-training for graph
representation learning. With hundreds of graph pre-training tasks proposed,
integrating knowledge acquired from multiple pre-training tasks has become a
popular research topic. In this paper, we identify two important collaborative
processes for this topic: (1) select: how to select an optimal task combination
from a given task pool based on their compatibility, and (2) weigh: how to
weigh the selected tasks based on their importance. While there currently has
been a lot of work focused on weighing, comparatively little effort has been
devoted to selecting. This paper proposes a novel instance-level framework for
integrating multiple graph pre-training tasks, Weigh And Select (WAS), where
the two collaborative processes, weighing and selecting, are combined by
decoupled siamese networks. Specifically, it first adaptively learns an optimal
combination of tasks for each instance from a given task pool, based on which a
customized instance-level task weighing strategy is learned. Extensive
experiments on 16 graph datasets across node-level and graph-level downstream
tasks have demonstrated that by combining a few simple but classical tasks, WAS
can achieve comparable performance to other leading counterparts. The code is
available at https://github.com/TianyuFan0504/WAS.