Mixing Classifiers to Alleviate the Accuracy-Robustness Trade-Off
Yatong Bai Brendon G. Anderson Somayeh Sojoudi
Machine learning models have recently found tremendous success in data-driven
control systems. However, standard learning models often suffer from an
accuracy-robustness trade-off, which is a limitation that must be overcome in
the control of safety-critical systems that require both high performance and
rigorous robustness guarantees. In this work, we build upon the recent "locally
biased smoothing" method to develop classifiers that simultaneously inherit
high accuracy from standard models and high robustness from robust models.
Specifically, we extend locally biased smoothing to the multi-class setting,
and then overcome its performance bottleneck by generalizing the formulation to
"mix" the outputs of a standard neural network and a robust neural network. We
prove that when the robustness of the robust base model is certifiable, within
a closed-form $\ell_p$ radius, no alteration or attack on an input can result
in misclassification of the mixed classifier; the proposed model inherits the
certified robustness. Moreover, we use numerical experiments on the CIFAR-10
benchmark dataset to verify that the mixed model noticeably improves the
accuracy-robustness trade-off.