 Discriminative training of linear classifiers usually proceeds in a supervised way, by means of an optimization algorithm that is given a training set with desired outputs and a loss function that measures the discrepancy between the classifier's outputs and the desired outputs.Thus, the learning algorithm solves an optimization problem of the form

where


Popular loss functions include the hinge loss (for linear SVMs) and the log loss (for linear logistic regression).If the regularization function is convex, then the above is a convex problem.