Separability and Scatteredness (S&S) Ratio-Based Efficient SVM
  Regularization Parameter, Kernel, and Kernel Parameter Selection
Mahdi Shamsi Soosan Beheshti
Support Vector Machine (SVM) is a robust machine learning algorithm with
broad applications in classification, regression, and outlier detection. SVM
requires tuning the regularization parameter (RP) which controls the model
capacity and the generalization performance. Conventionally, the optimum RP is
found by comparison of a range of values through the Cross-Validation (CV)
procedure. In addition, for non-linearly separable data, the SVM uses kernels
where a set of kernels, each with a set of parameters, denoted as a grid of
kernels, are considered. The optimal choice of RP and the grid of kernels is
through the grid-search of CV. By stochastically analyzing the behavior of the
regularization parameter, this work shows that the SVM performance can be
modeled as a function of separability and scatteredness (S&S) of the data.
Separability is a measure of the distance between classes, and scatteredness is
the ratio of the spread of data points. In particular, for the hinge loss cost
function, an S&S ratio-based table provides the optimum RP. The S&S ratio is a
powerful value that can automatically detect linear or non-linear separability
before using the SVM algorithm. The provided S&S ratio-based table can also
provide the optimum kernel and its parameters before using the SVM algorithm.
Consequently, the computational complexity of the CV grid-search is reduced to
only one time use of the SVM. The simulation results on the real dataset
confirm the superiority and efficiency of the proposed approach in the sense of
computational complexity over the grid-search CV method.