On Subagging Boosted Probit Model Trees
Tian Qin Wei-Min Huang
With the insight of variance-bias decomposition, we design a new hybrid
bagging-boosting algorithm named SBPMT for classification problems. For the
boosting part of SBPMT, we propose a new tree model called Probit Model Tree
(PMT) as base classifiers in AdaBoost procedure. For the bagging part, instead
of subsampling from the dataset at each step of boosting, we perform boosted
PMTs on each subagged dataset and combine them into a powerful "committee",
which can be viewed an incomplete U-statistic. Our theoretical analysis shows
that (1) SBPMT is consistent under certain assumptions, (2) Increase the
subagging times can reduce the generalization error of SBPMT to some extent and
(3) Large number of ProbitBoost iterations in PMT can benefit the performance
of SBPMT with fewer steps in the AdaBoost part. Those three properties are
verified by a famous simulation designed by Mease and Wyner (2008). The last
two points also provide a useful guidance in model tuning. A comparison of
performance with other state-of-the-art classification methods illustrates that
the proposed SBPMT algorithm has competitive prediction power in general and
performs significantly better in some cases.