SKIP: Skill-Localized Prompt Tuning for Inference Speed Boost-Up
Nakyeong Yang Junseok Kim Jiwon Moon Yunah Jang Kyomin Jung
Prompt-tuning methods have shown comparable performance as
parameter-efficient fine-tuning (PEFT) methods in various natural language
understanding tasks. However, existing prompt tuning methods still utilize the
entire model architecture; thus, they fail to accelerate inference speed in the
application. In this paper, we propose a novel approach called SKIll-localized
Prompt tuning (SKIP), which is extremely efficient in inference time. Our
method significantly enhances inference efficiency by investigating and
utilizing a skill-localized subnetwork in a language model. Surprisingly, our
method improves the inference speed up to 160% while pruning 52% of the
parameters. Furthermore, we demonstrate that our method is applicable across
various transformer-based architectures, thereby confirming its practicality
and scalability.