A Differentially Private Weighted Empirical Risk Minimization Procedure
  and its Application to Outcome Weighted Learning
Spencer Giddens Yiwang Zhou Kevin R. Krull Tara M. Brinkman Peter X. K. Song Fang Liu
It is commonplace to use data containing personal information to build
predictive models in the framework of empirical risk minimization (ERM). While
these models can be highly accurate in prediction, results obtained from these
models with the use of sensitive data may be susceptible to privacy attacks.
Differential privacy (DP) is an appealing framework for addressing such data
privacy issues by providing mathematically provable bounds on the privacy loss
incurred when releasing information from sensitive data. Previous work has
primarily concentrated on applying DP to unweighted ERM. We consider an
important generalization to weighted ERM (wERM). In wERM, each individual's
contribution to the objective function can be assigned varying weights. In this
context, we propose the first differentially private wERM algorithm, backed by
a rigorous theoretical proof of its DP guarantees under mild regularity
conditions. Extending the existing DP-ERM procedures to wERM paves a path to
deriving privacy-preserving learning methods for individualized treatment
rules, including the popular outcome weighted learning (OWL). We evaluate the
performance of the DP-wERM application to OWL in a simulation study and in a
real clinical trial of melatonin for sleep health. All empirical results
demonstrate the viability of training OWL models via wERM with DP guarantees
while maintaining sufficiently useful model performance. Therefore, we
recommend practitioners consider implementing the proposed privacy-preserving
OWL procedure in real-world scenarios involving sensitive data.