Random Matrix Analysis to Balance between Supervised and Unsupervised
  Learning under the Low Density Separation Assumption
Vasilii Feofanov Malik Tiomoko Aladin Virmaux
We propose a theoretical framework to analyze semi-supervised classification
under the low density separation assumption in a high-dimensional regime. In
particular, we introduce QLDS, a linear classification model, where the low
density separation assumption is implemented via quadratic margin maximization.
The algorithm has an explicit solution with rich theoretical properties, and we
show that particular cases of our algorithm are the least-square support vector
machine in the supervised case, the spectral clustering in the fully
unsupervised regime, and a class of semi-supervised graph-based approaches. As
such, QLDS establishes a smooth bridge between these supervised and
unsupervised learning methods. Using recent advances in the random matrix
theory, we formally derive a theoretical evaluation of the classification error
in the asymptotic regime. As an application, we derive a hyperparameter
selection policy that finds the best balance between the supervised and the
unsupervised terms of our learning criterion. Finally, we provide extensive
illustrations of our framework, as well as an experimental study on several
benchmarks to demonstrate that QLDS, while being computationally more
efficient, improves over cross-validation for hyperparameter selection,
indicating a high promise of the usage of random matrix theory for
semi-supervised model selection.