A framework for paired-sample hypothesis testing for high-dimensional
  data
Ioannis Bargiotas Argyris Kalogeratos Nicolas Vayatis
The standard paired-sample testing approach in the multidimensional setting
applies multiple univariate tests on the individual features, followed by
p-value adjustments. Such an approach suffers when the data carry numerous
features. A number of studies have shown that classification accuracy can be
seen as a proxy for two-sample testing. However, neither theoretical
foundations nor practical recipes have been proposed so far on how this
strategy could be extended to multidimensional paired-sample testing. In this
work, we put forward the idea that scoring functions can be produced by the
decision rules defined by the perpendicular bisecting hyperplanes of the line
segments connecting each pair of instances. Then, the optimal scoring function
can be obtained by the pseudomedian of those rules, which we estimate by
extending naturally the Hodges-Lehmann estimator. We accordingly propose a
framework of a two-step testing procedure. First, we estimate the bisecting
hyperplanes for each pair of instances and an aggregated rule derived through
the Hodges-Lehmann estimator. The paired samples are scored by this aggregated
rule to produce a unidimensional representation. Second, we perform a Wilcoxon
signed-rank test on the obtained representation. Our experiments indicate that
our approach has substantial performance gains in testing accuracy compared to
the traditional multivariate and multiple testing, while at the same time
estimates each feature's contribution to the final result.