Binary Feature Mask Optimization for Feature Selection
Mehmet E. Lorasdagi Mehmet Y. Turali Ali T. Koc Suleyman S. Kozat
We investigate feature selection problem for generic machine learning (ML)
models. We introduce a novel framework that selects features considering the
predictions of the model. Our framework innovates by using a novel feature
masking approach to eliminate the features during the selection process,
instead of completely removing them from the dataset. This allows us to use the
same ML model during feature selection, unlike other feature selection methods
where we need to train the ML model again as the dataset has different
dimensions on each iteration. We obtain the mask operator using the predictions
of the ML model, which offers a comprehensive view on the subsets of the
features essential for the predictive performance of the model. A variety of
approaches exist in the feature selection literature. However, no study has
introduced a training-free framework for a generic ML model to select features
while considering the importance of the feature subsets as a whole, instead of
focusing on the individual features. We demonstrate significant performance
improvements on the real-life datasets under different settings using LightGBM
and Multi-Layer Perceptron as our ML models. Additionally, we openly share the
implementation code for our methods to encourage the research and the
contributions in this area.