Tackling Negative Transfer on Graphs
Zehong Wang Zheyuan Zhang Chuxu Zhang Yanfang Ye
Transfer learning aims to boost the learning on the target task leveraging
knowledge learned from other relevant tasks. However, when the source and
target are not closely related, the learning performance may be adversely
affected, a phenomenon known as negative transfer. In this paper, we
investigate the negative transfer in graph transfer learning, which is
important yet underexplored. We reveal that, unlike image or text, negative
transfer commonly occurs in graph-structured data, even when source and target
graphs share semantic similarities. Specifically, we identify that structural
differences significantly amplify the dissimilarities in the node embeddings
across graphs. To mitigate this, we bring a new insight: for semantically
similar graphs, although structural differences lead to significant
distribution shift in node embeddings, their impact on subgraph embeddings
could be marginal. Building on this insight, we introduce two effective yet
elegant methods, Subgraph Pooling (SP) and Subgraph Pooling++ (SP++), that
transfer subgraph-level knowledge across graphs. We theoretically analyze the
role of SP in reducing graph discrepancy and conduct extensive experiments to
evaluate its superiority under various settings. Our code and datasets are
available at: https://github.com/Zehong-Wang/Subgraph-Pooling.