Augment then Smooth: Reconciling Differential Privacy with Certified
  Robustness
Jiapeng Wu Atiyeh Ashari Ghomi David Glukhov Jesse C. Cresswell Franziska Boenisch Nicolas Papernot
Machine learning models are susceptible to a variety of attacks that can
erode trust in their deployment. These threats include attacks against the
privacy of training data and adversarial examples that jeopardize model
accuracy. Differential privacy and randomized smoothing are effective defenses
that provide certifiable guarantees for each of these threats, however, it is
not well understood how implementing either defense impacts the other. In this
work, we argue that it is possible to achieve both privacy guarantees and
certified robustness simultaneously. We provide a framework called DP-CERT for
integrating certified robustness through randomized smoothing into
differentially private model training. For instance, compared to differentially
private stochastic gradient descent on CIFAR10, DP-CERT leads to a 12-fold
increase in certified accuracy and a 10-fold increase in the average certified
radius at the expense of a drop in accuracy of 1.2%. Through in-depth
per-sample metric analysis, we show that the certified radius correlates with
the local Lipschitz constant and smoothness of the loss surface. This provides
a new way to diagnose when private models will fail to be robust.