Riemann-Lebesgue Forest for Regression
Tian Qin Wei-Min Huang
We propose a novel ensemble method called Riemann-Lebesgue Forest (RLF) for
regression. The core idea of RLF is to mimic the way how a measurable function
can be approximated by partitioning its range into a few intervals. With this
idea in mind, we develop a new tree learner named Riemann-Lebesgue Tree which
has a chance to split the node from response $Y$ or a direction in feature
space $\mathbf{X}$ at each non-terminal node. We generalize the asymptotic
performance of RLF under different parameter settings mainly through Hoeffding
decomposition \cite{Vaart} and Stein's method \cite{Chen2010NormalAB}. When the
underlying function $Y=f(\mathbf{X})$ follows an additive regression model, RLF
is consistent with the argument from \cite{Scornet2014ConsistencyOR}. The
competitive performance of RLF against original random forest
\cite{Breiman2001RandomF} is demonstrated by experiments in simulation data and
real world datasets.