Modified Genetic Algorithm for Feature Selection and Hyper Parameter
  Optimization: Case of XGBoost in Spam Prediction
Nazeeh Ghatasheh Ismail Altaharwa Khaled Aldebei
Recently, spam on online social networks has attracted attention in the
research and business world. Twitter has become the preferred medium to spread
spam content. Many research efforts attempted to encounter social networks
spam. Twitter brought extra challenges represented by the feature space size,
and imbalanced data distributions. Usually, the related research works focus on
part of these main challenges or produce black-box models. In this paper, we
propose a modified genetic algorithm for simultaneous dimensionality reduction
and hyper parameter optimization over imbalanced datasets. The algorithm
initialized an eXtreme Gradient Boosting classifier and reduced the features
space of tweets dataset; to generate a spam prediction model. The model is
validated using a 50 times repeated 10-fold stratified cross-validation, and
analyzed using nonparametric statistical tests. The resulted prediction model
attains on average 82.32\% and 92.67\% in terms of geometric mean and accuracy
respectively, utilizing less than 10\% of the total feature space. The
empirical results show that the modified genetic algorithm outperforms $Chi^2$
and $PCA$ feature selection methods. In addition, eXtreme Gradient Boosting
outperforms many machine learning algorithms, including BERT-based deep
learning model, in spam prediction. Furthermore, the proposed approach is
applied to SMS spam modeling and compared to related works.