Benchmarking state-of-the-art gradient boosting algorithms for
  classification
Piotr Florek Adam Zagdański
This work explores the use of gradient boosting in the context of
classification. Four popular implementations, including original GBM algorithm
and selected state-of-the-art gradient boosting frameworks (i.e. XGBoost,
LightGBM and CatBoost), have been thoroughly compared on several publicly
available real-world datasets of sufficient diversity. In the study, special
emphasis was placed on hyperparameter optimization, specifically comparing two
tuning strategies, i.e. randomized search and Bayesian optimization using the
Tree-stuctured Parzen Estimator. The performance of considered methods was
investigated in terms of common classification accuracy metrics as well as
runtime and tuning time. Additionally, obtained results have been validated
using appropriate statistical testing. An attempt was made to indicate a
gradient boosting variant showing the right balance between effectiveness,
reliability and ease of use.