Text-CRS: A Generalized Certified Robustness Framework against Textual
  Adversarial Attacks
Xinyu Zhang Hanbin Hong Yuan Hong Peng Huang Binghui Wang Zhongjie Ba Kui Ren
The language models, especially the basic text classification models, have
been shown to be susceptible to textual adversarial attacks such as synonym
substitution and word insertion attacks. To defend against such attacks, a
growing body of research has been devoted to improving the model robustness.
However, providing provable robustness guarantees instead of empirical
robustness is still widely unexplored. In this paper, we propose Text-CRS, a
generalized certified robustness framework for natural language processing
(NLP) based on randomized smoothing. To our best knowledge, existing certified
schemes for NLP can only certify the robustness against $\ell_0$ perturbations
in synonym substitution attacks. Representing each word-level adversarial
operation (i.e., synonym substitution, word reordering, insertion, and
deletion) as a combination of permutation and embedding transformation, we
propose novel smoothing theorems to derive robustness bounds in both
permutation and embedding space against such adversarial operations. To further
improve certified accuracy and radius, we consider the numerical relationships
between discrete words and select proper noise distributions for the randomized
smoothing. Finally, we conduct substantial experiments on multiple language
models and datasets. Text-CRS can address all four different word-level
adversarial operations and achieve a significant accuracy improvement. We also
provide the first benchmark on certified accuracy and radius of four word-level
operations, besides outperforming the state-of-the-art certification against
synonym substitution attacks.