SEA: Shareable and Explainable Attribution for Query-based Black-box
  Attacks
Yue Gao Ilia Shumailov Kassem Fawaz
Machine Learning (ML) systems are vulnerable to adversarial examples,
particularly those from query-based black-box attacks. Despite various efforts
to detect and prevent such attacks, there is a need for a more comprehensive
approach to logging, analyzing, and sharing evidence of attacks. While classic
security benefits from well-established forensics and intelligence sharing,
Machine Learning is yet to find a way to profile its attackers and share
information about them. In response, this paper introduces SEA, a novel ML
security system to characterize black-box attacks on ML systems for forensic
purposes and to facilitate human-explainable intelligence sharing. SEA
leverages the Hidden Markov Models framework to attribute the observed query
sequence to known attacks. It thus understands the attack's progression rather
than just focusing on the final adversarial examples. Our evaluations reveal
that SEA is effective at attack attribution, even on their second occurrence,
and is robust to adaptive strategies designed to evade forensics analysis.
Interestingly, SEA's explanations of the attack behavior allow us even to
fingerprint specific minor implementation bugs in attack libraries. For
example, we discover that the SignOPT and Square attacks implementation in ART
v1.14 sends over 50% specific zero difference queries. We thoroughly evaluate
SEA on a variety of settings and demonstrate that it can recognize the same
attack's second occurrence with 90+% Top-1 and 95+% Top-3 accuracy.