Discriminative learning according to the loss function

Many machine learning methods reduce to the constraint miminization problem. Examples include the maximum entropy model, the conditional random field, the support vector machines, the maximum margin markov network and the margin infused relaxed algorithm, just to name a few. Most of the problems can be solved by lagrangian multiplier.

For the largin margin learning methods, the learner tries to find the parameters that maximises the score difference between the correct answer and all the incorrect answers. It's the same for both structural prediction and simple classification. In the implementation, the margin (i.e. score difference) must be defined. It can be simply 1, as the classic SVM does, or other functions, especially in structural prediction. Here is a good tutorial.

The interesting idea from the tutorial is viewing many discriminative learning algorithms as special loss minimisation problems - with different loss functions.