UOR: Universal Backdoor Attacks on Pre-trained Language Models
Wei Du Peixuan Li Boqun Li Haodong Zhao Gongshen Liu
Backdoors implanted in pre-trained language models (PLMs) can be transferred
to various downstream tasks, which exposes a severe security threat. However,
most existing backdoor attacks against PLMs are un-targeted and task-specific.
Few targeted and task-agnostic methods use manually pre-defined triggers and
output representations, which prevent the attacks from being more effective and
general. In this paper, we first summarize the requirements that a more
threatening backdoor attack against PLMs should satisfy, and then propose a new
backdoor attack method called UOR, which breaks the bottleneck of the previous
approach by turning manual selection into automatic optimization. Specifically,
we define poisoned supervised contrastive learning which can automatically
learn the more uniform and universal output representations of triggers for
various PLMs. Moreover, we use gradient search to select appropriate trigger
words which can be adaptive to different PLMs and vocabularies. Experiments
show that our method can achieve better attack performance on various text
classification tasks compared to manual methods. Further, we tested our method
on PLMs with different architectures, different usage paradigms, and more
difficult tasks, which demonstrated the universality of our method.