Conformalised data synthesis with statistical quality guarantees
Julia A. Meister Khuong An Nguyen
With the proliferation of ever more complicated Deep Learning architectures,
data synthesis is a highly promising technique to address the demand of
data-hungry models. However, reliably assessing the quality of a 'synthesiser'
model's output is an open research question with significant associated risks
for high-stake domains. To address this challenge, we have designed a unique
confident data synthesis algorithm that introduces statistical confidence
guarantees through a novel extension of the Conformal Prediction framework. We
support our proposed algorithm with theoretical proofs and an extensive
empirical evaluation of five benchmark datasets. To show our approach's
versatility on ubiquitous real-world challenges, the datasets were carefully
selected for their variety of difficult characteristics: low sample count,
class imbalance and non-separability, and privacy-sensitive data. In all
trials, training sets extended with our confident synthesised data performed at
least as well as the original, and frequently significantly improved Deep
Learning performance by up to +65% F1-score.