The fact that a classifier is "Bayesian" does not endow it with any
particular advantage versus SVMs. The key question is the nature of
the hypothesis class. SVMs (with kernels) explore a very interesting
and flexible hypothesis class whose complexity can easily be tuned to
the amount and complexity of the data; most Bayesian classifiers
search classes that are much less flexible. So in many cases, SVMs
will out-perform Bayesian classifiers. However, Bayesian model
averaging and SVM margin maximization are both just mechanisms, and
their behavior depends on the prior (in the Bayesian case) and the
choice of kernel and other parameters (in the SVM case). And of
course, even more important than the regularizer and concept class is
the choice of input features, the level of noise in the data, and the
amount of training data available.
--
Thomas G. Dietterich Voice: 541-737-5559
Department of Computer Science FAX: 541-737-3014
Dearborn Hall 102 URL: http://www.cs.orst.edu/~tgd
Oregon State University
Corvallis, OR 97331-3102
|