Variable selection for Naïve Bayes classification
Rafael Blanquero Emilio Carrizosa Pepa Ramírez-Cobo M. Remedios Sillero-Denamiel
The Na\"ive Bayes has proven to be a tractable and efficient method for
classification in multivariate analysis. However, features are usually
correlated, a fact that violates the Na\"ive Bayes' assumption of conditional
independence, and may deteriorate the method's performance. Moreover, datasets
are often characterized by a large number of features, which may complicate the
interpretation of the results as well as slow down the method's execution.
  In this paper we propose a sparse version of the Na\"ive Bayes classifier
that is characterized by three properties. First, the sparsity is achieved
taking into account the correlation structure of the covariates. Second,
different performance measures can be used to guide the selection of features.
Third, performance constraints on groups of higher interest can be included.
Our proposal leads to a smart search, which yields competitive running times,
whereas the flexibility in terms of performance measure for classification is
integrated. Our findings show that, when compared against well-referenced
feature selection approaches, the proposed sparse Na\"ive Bayes obtains
competitive results regarding accuracy, sparsity and running times for balanced
datasets. In the case of datasets with unbalanced (or with different
importance) classes, a better compromise between classification rates for the
different classes is achieved.