Enhance Robustness of Language Models Against Variation Attack through
  Graph Integration
Zi Xiong Lizhi Qing Yangyang Kang Jiawei Liu Hongsong Li Changlong Sun Xiaozhong Liu Wei Lu
The widespread use of pre-trained language models (PLMs) in natural language
processing (NLP) has greatly improved performance outcomes. However, these
models' vulnerability to adversarial attacks (e.g., camouflaged hints from drug
dealers), particularly in the Chinese language with its rich character
diversity/variation and complex structures, hatches vital apprehension. In this
study, we propose a novel method, CHinese vAriatioN Graph Enhancement (CHANGE),
to increase the robustness of PLMs against character variation attacks in
Chinese content. CHANGE presents a novel approach for incorporating a Chinese
character variation graph into the PLMs. Through designing different
supplementary tasks utilizing the graph structure, CHANGE essentially enhances
PLMs' interpretation of adversarially manipulated text. Experiments conducted
in a multitude of NLP tasks show that CHANGE outperforms current language
models in combating against adversarial attacks and serves as a valuable
contribution to robust language model research. These findings contribute to
the groundwork on robust language models and highlight the substantial
potential of graph-guided pre-training strategies for real-world applications.