Correlation and Unintended Biases on Univariate and Multivariate
  Decision Trees
Mattia Setzu Salvatore Ruggieri
Decision Trees are accessible, interpretable, and well-performing
classification models. A plethora of variants with increasing expressiveness
has been proposed in the last forty years. We contrast the two families of
univariate DTs, whose split functions partition data through axis-parallel
hyperplanes, and multivariate DTs, whose splits instead partition data through
oblique hyperplanes. The latter include the former, hence multivariate DTs are
in principle more powerful. Surprisingly enough, however, univariate DTs
consistently show comparable performances in the literature. We analyze the
reasons behind this, both with synthetic and real-world benchmark datasets. Our
research questions test whether the pre-processing phase of removing
correlation among features in datasets has an impact on the relative
performances of univariate vs multivariate DTs. We find that existing benchmark
datasets are likely biased towards favoring univariate DTs.