Calibration Attack: A Framework For Adversarial Attacks Targeting
  Calibration
Stephen Obadinma Xiaodan Zhu Hongyu Guo
We introduce a new framework of adversarial attacks, named calibration
attacks, in which the attacks are generated and organized to trap victim models
to be miscalibrated without altering their original accuracy, hence seriously
endangering the trustworthiness of the models and any decision-making based on
their confidence scores. Specifically, we identify four novel forms of
calibration attacks: underconfidence attacks, overconfidence attacks, maximum
miscalibration attacks, and random confidence attacks, in both the black-box
and white-box setups. We then test these new attacks on typical victim models
with comprehensive datasets, demonstrating that even with a relatively low
number of queries, the attacks can create significant calibration mistakes. We
further provide detailed analyses to understand different aspects of
calibration attacks. Building on that, we investigate the effectiveness of
widely used adversarial defences and calibration methods against these types of
attacks, which then inspires us to devise two novel defences against such
calibration attacks.