Accuracy of TextFooler black box adversarial attacks on 01 loss sign
  activation neural network ensemble
Yunzhe Xue Usman Roshan
Recent work has shown the defense of 01 loss sign activation neural networks
against image classification adversarial attacks. A public challenge to attack
the models on CIFAR10 dataset remains undefeated. We ask the following question
in this study: are 01 loss sign activation neural networks hard to deceive with
a popular black box text adversarial attack program called TextFooler? We study
this question on four popular text classification datasets: IMDB reviews, Yelp
reviews, MR sentiment classification, and AG news classification. We find that
our 01 loss sign activation network is much harder to attack with TextFooler
compared to sigmoid activation cross entropy and binary neural networks. We
also study a 01 loss sign activation convolutional neural network with a novel
global pooling step specific to sign activation networks. With this new
variation we see a significant gain in adversarial accuracy rendering
TextFooler practically useless against it. We make our code freely available at
\url{https://github.com/zero-one-loss/wordcnn01} and
\url{https://github.com/xyzacademic/mlp01example}. Our work here suggests that
01 loss sign activation networks could be further developed to create fool
proof models against text adversarial attacks.