Smooth Ranking SVM via Cutting-Plane Method
Erhan Can Ozcan Berk Görgülü Mustafa G. Baydogan Ioannis Ch. Paschalidis
The most popular classification algorithms are designed to maximize
classification accuracy during training. However, this strategy may fail in the
presence of class imbalance since it is possible to train models with high
accuracy by overfitting to the majority class. On the other hand, the Area
Under the Curve (AUC) is a widely used metric to compare classification
performance of different algorithms when there is a class imbalance, and
various approaches focusing on the direct optimization of this metric during
training have been proposed. Among them, SVM-based formulations are especially
popular as this formulation allows incorporating different regularization
strategies easily. In this work, we develop a prototype learning approach that
relies on cutting-plane method, similar to Ranking SVM, to maximize AUC. Our
algorithm learns simpler models by iteratively introducing cutting planes, thus
overfitting is prevented in an unconventional way. Furthermore, it penalizes
the changes in the weights at each iteration to avoid large jumps that might be
observed in the test performance, thus facilitating a smooth learning process.
Based on the experiments conducted on 73 binary classification datasets, our
method yields the best test AUC in 25 datasets among its relevant competitors.