DP-TBART: A Transformer-based Autoregressive Model for Differentially
  Private Tabular Data Generation
Rodrigo Castellon Achintya Gopal Brian Bloniarz David Rosenberg
The generation of synthetic tabular data that preserves differential privacy
is a problem of growing importance. While traditional marginal-based methods
have achieved impressive results, recent work has shown that deep
learning-based approaches tend to lag behind. In this work, we present
Differentially-Private TaBular AutoRegressive Transformer (DP-TBART), a
transformer-based autoregressive model that maintains differential privacy and
achieves performance competitive with marginal-based methods on a wide variety
of datasets, capable of even outperforming state-of-the-art methods in certain
settings. We also provide a theoretical framework for understanding the
limitations of marginal-based approaches and where deep learning-based
approaches stand to contribute most. These results suggest that deep
learning-based techniques should be considered as a viable alternative to
marginal-based methods in the generation of differentially private synthetic
tabular data.