Combining propensity score methods with variational autoencoders for
  generating synthetic data in presence of latent sub-groups
Kiana Farhadyar Federico Bonofiglio Maren Hackenberg Daniela Zoeller Harald Binder
In settings requiring synthetic data generation based on a clinical cohort,
e.g., due to data protection regulations, heterogeneity across individuals
might be a nuisance that we need to control or faithfully preserve. The sources
of such heterogeneity might be known, e.g., as indicated by sub-groups labels,
or might be unknown and thus reflected only in properties of distributions,
such as bimodality or skewness. We investigate how such heterogeneity can be
preserved and controlled when obtaining synthetic data from variational
autoencoders (VAEs), i.e., a generative deep learning technique that utilizes a
low-dimensional latent representation. To faithfully reproduce unknown
heterogeneity reflected in marginal distributions, we propose to combine VAEs
with pre-transformations. For dealing with known heterogeneity due to
sub-groups, we complement VAEs with models for group membership, specifically
from propensity score regression. The evaluation is performed with a realistic
simulation design that features sub-groups and challenging marginal
distributions. The proposed approach faithfully recovers the latter, compared
to synthetic data approaches that focus purely on marginal distributions.
Propensity scores add complementary information, e.g., when visualized in the
latent space, and enable sampling of synthetic data with or without sub-group
specific characteristics. We also illustrate the proposed approach with real
data from an international stroke trial that exhibits considerable distribution
differences between study sites, in addition to bimodality. These results
indicate that describing heterogeneity by statistical approaches, such as
propensity score regression, might be more generally useful for complementing
generative deep learning for obtaining synthetic data that faithfully reflects
structure from clinical cohorts.