Bounding the Excess Risk for Linear Models Trained on
  Marginal-Preserving, Differentially-Private, Synthetic Data
Yvonne Zhou Mingyu Liang Ivan Brugere Dana Dachman-Soled Danial Dervovic Antigoni Polychroniadou Min Wu
The growing use of machine learning (ML) has raised concerns that an ML model
may reveal private information about an individual who has contributed to the
training dataset. To prevent leakage of sensitive data, we consider using
differentially-private (DP), synthetic training data instead of real training
data to train an ML model. A key desirable property of synthetic data is its
ability to preserve the low-order marginals of the original distribution. Our
main contribution comprises novel upper and lower bounds on the excess
empirical risk of linear models trained on such synthetic data, for continuous
and Lipschitz loss functions. We perform extensive experimentation alongside
our theoretical results.