Generative models and discriminative models - comparison

Given a set of samples (x1, y1) ... (xk, yk) , try to find a model behind them by supervised learning.

Both the generative and the discriminative models assume that there is an underlying probability distribution P(x, y) that generates the samples, however,

- The generative models try to find exactly the distribution P(x, y)
examples: Bayesian models
structural prediction: find a probabilistic model that simplifies P(x,y), usually by the chain rule / Bayes theorum / Independence assumption.

- The discriminative models find some distribution with low error rate on the training examples
examples: support vector machines
structural prediction: find a feature vector representation of the structures of Y.

Daume III (2006) gave an example where a generative model makes much worse error expection than a discriminative model (chap 3).