Subgraph-level Universal Prompt Tuning
Junhyun Lee Wooseong Yang Jaewoo Kang
In the evolving landscape of machine learning, the adaptation of pre-trained
models through prompt tuning has become increasingly prominent. This trend is
particularly observable in the graph domain, where diverse pre-training
strategies present unique challenges in developing effective prompt-based
tuning methods for graph neural networks. Previous approaches have been
limited, focusing on specialized prompting functions tailored to models with
edge prediction pre-training tasks. These methods, however, suffer from a lack
of generalizability across different pre-training strategies. Recently, a
simple prompt tuning method has been designed for any pre-training strategy,
functioning within the input graph's feature space. This allows it to
theoretically emulate any type of prompting function, thereby significantly
increasing its versatility for a range of downstream applications.
Nevertheless, the capacity of such simple prompts to fully grasp the complex
contexts found in graphs remains an open question, necessitating further
investigation. Addressing this challenge, our work introduces the
Subgraph-level Universal Prompt Tuning (SUPT) approach, focusing on the
detailed context within subgraphs. In SUPT, prompt features are assigned at the
subgraph-level, preserving the method's universal capability. This requires
extremely fewer tuning parameters than fine-tuning-based methods, outperforming
them in 42 out of 45 full-shot scenario experiments with an average improvement
of over 2.5%. In few-shot scenarios, it excels in 41 out of 45 experiments,
achieving an average performance increase of more than 6.6%.