See also Cristianini and Shawe-Taylor's book, Section 6.1.2 where C = 1/R^2
is suggested for the case of quadratically penalized errors. (Similarly
motivated from a generalization error bound.)
----- Original Message -----
From: "Koby Crammer" <[log in to unmask]>
To: <[log in to unmask]>
Sent: Friday, May 28, 2004 4:24 AM
Subject: Re: origin of norm-based default regularization parameter in SVM
Light?
> Hi,
>
> Well, I have a hint which can help. There is a new bound for the number of
the
> Mistakes the Perceptron algorithm makes. In terms of SVM the bound is
> proportional to (for all w)
> (1/2)||w||^2 R^2 + \sum_i \max\{0, 1-y_i(w*x_i)\}
> where it is assumed that all the examples are in a ball of radius R around
the
> origin.
>
> If you take the factor R^2 out of the bound you get the SVM optimization
> problem, where C=1/R^2 ; a max operator is used rather than the mean
operator
> you asked about. I am not sure if the bound still holds if we replace the
> max with a mean. In practice, the mean is more stable than the max.
(Although I would try to use
> also the median).
>
> Regards, Koby
>
>
> ============================================
> Koby Crammer [log in to unmask]
>
> http://www.cs.huji.ac.il/~kobics
> ============================================
> On Thu, 27 May 2004, Dave Lewis wrote:
>
> | Hi - SVM Light has as the default value of its regularization parameter
> | (tradeoff between training error and margin) the reciprocal of the mean
> | 2-norm of the training examples. Does anyone know what the statistical
> | justification for this particular choice is? I can't seem to find this
in
> | Thorsten Joachim's papers.
> |
> | Regards, Dave
> | http://www.DavidDLewis.com
> |
|