Generative Forests
Richard Nock Mathieu Guillame-Bert
Tabular data represents one of the most prevalent form of data. When it comes
to data generation, many approaches would learn a density for the data
generation process, but would not necessarily end up with a sampler, even less
so being exact with respect to the underlying density. A second issue is on
models: while complex modeling based on neural nets thrives in image or text
generation (etc.), less is known for powerful generative models on tabular
data. A third problem is the visible chasm on tabular data between training
algorithms for supervised learning with remarkable properties (e.g. boosting),
and a comparative lack of guarantees when it comes to data generation. In this
paper, we tackle the three problems, introducing new tree-based generative
models convenient for density modeling and tabular data generation that improve
on modeling capabilities of recent proposals, and a training algorithm which
simplifies the training setting of previous approaches and displays
boosting-compliant convergence. This algorithm has the convenient property to
rely on a supervised training scheme that can be implemented by a few tweaks to
the most popular induction scheme for decision tree induction with two classes.
Experiments are provided on missing data imputation and comparing generated
data to real data, displaying the quality of the results obtained by our
approach, in particular against state of the art.