Majorization-Minimization for sparse SVMs
Alessandro Benfenati Emilie Chouzenoux Giorgia Franchini Salla Latva-Aijo Dominik Narnhofer Jean-Christophe Pesquet Sebastian J. Scott Mahsa Yousefi
Several decades ago, Support Vector Machines (SVMs) were introduced for
performing binary classification tasks, under a supervised framework. Nowadays,
they often outperform other supervised methods and remain one of the most
popular approaches in the machine learning arena. In this work, we investigate
the training of SVMs through a smooth sparse-promoting-regularized squared
hinge loss minimization. This choice paves the way to the application of quick
training methods built on majorization-minimization approaches, benefiting from
the Lipschitz differentiabililty of the loss function. Moreover, the proposed
approach allows us to handle sparsity-preserving regularizers promoting the
selection of the most significant features, so enhancing the performance.
Numerical tests and comparisons conducted on three different datasets
demonstrate the good performance of the proposed methodology in terms of
qualitative metrics (accuracy, precision, recall, and F 1 score) as well as
computational cost.