Fast Genetic Algorithm for feature selection -- A qualitative
  approximation approach
Mohammed Ghaith Altarabichi Sławomir Nowaczyk Sepideh Pashami Peyman Sheikholharam Mashhadi
Evolutionary Algorithms (EAs) are often challenging to apply in real-world
settings since evolutionary computations involve a large number of evaluations
of a typically expensive fitness function. For example, an evaluation could
involve training a new machine learning model. An approximation (also known as
meta-model or a surrogate) of the true function can be used in such
applications to alleviate the computation cost. In this paper, we propose a
two-stage surrogate-assisted evolutionary approach to address the computational
issues arising from using Genetic Algorithm (GA) for feature selection in a
wrapper setting for large datasets. We define 'Approximation Usefulness' to
capture the necessary conditions to ensure correctness of the EA computations
when an approximation is used. Based on this definition, we propose a procedure
to construct a lightweight qualitative meta-model by the active selection of
data instances. We then use a meta-model to carry out the feature selection
task. We apply this procedure to the GA-based algorithm CHC (Cross generational
elitist selection, Heterogeneous recombination and Cataclysmic mutation) to
create a Qualitative approXimations variant, CHCQX. We show that CHCQX
converges faster to feature subset solutions of significantly higher accuracy
(as compared to CHC), particularly for large datasets with over 100K instances.
We also demonstrate the applicability of the thinking behind our approach more
broadly to Swarm Intelligence (SI), another branch of the Evolutionary
Computation (EC) paradigm with results of PSOQX, a qualitative approximation
adaptation of the Particle Swarm Optimization (PSO) method. A GitHub repository
with the complete implementation is available.