How do humans perceive adversarial text? A reality check on the validity
  and naturalness of word-based adversarial attacks
Salijona Dyrmishi Salah Ghamizi Maxime Cordy
Natural Language Processing (NLP) models based on Machine Learning (ML) are
susceptible to adversarial attacks -- malicious algorithms that imperceptibly
modify input text to force models into making incorrect predictions. However,
evaluations of these attacks ignore the property of imperceptibility or study
it under limited settings. This entails that adversarial perturbations would
not pass any human quality gate and do not represent real threats to
human-checked NLP systems. To bypass this limitation and enable proper
assessment (and later, improvement) of NLP model robustness, we have surveyed
378 human participants about the perceptibility of text adversarial examples
produced by state-of-the-art methods. Our results underline that existing text
attacks are impractical in real-world scenarios where humans are involved. This
contrasts with previous smaller-scale human studies, which reported overly
optimistic conclusions regarding attack success. Through our work, we hope to
position human perceptibility as a first-class success criterion for text
attacks, and provide guidance for research to build effective attack algorithms
and, in turn, design appropriate defence mechanisms.