Hello,
A good question.
'covariance':
A reference for this is Numerical Recipes, chapter 15.5 (Nonlinear models)
in the second edition (I think).
The standard Levenberg-Marquardt nonlinear fitting algorithm (which
is what we use) provides an estimate of the so-called covariance matrix.
This includes not only estimates of the variance (so also the standard
deviation) of the parameters (along the diagonal) but also the
cross-correlations. The covariance estimate we use completely ignores the
latter (so sort of assumes the parameter estimates are independent) and
just uses the diagonal values. Even if we didn't ignore the off-diagonal
terms apparently this is only a valid estimate if the distributions are
"normal" (gaussian).
'bootstrap':
A reference for this is:
Bootstrap Methods for Standard Errors, Confidence Intervals and Other
Measures of Statistical Accuracy
B. Efron and R. Tibshirani
Statistical Science, 1986, Vol. 1, No. 1, 54-77
What this does is repeatedly sample the data points (so, the (x,y) pairs)
over and over again. But with repeats allowed. So if you have 4 data
points, say, (x1, y1), (x2, y2), (x3, y3), (x4, y4) then the parameters
themselves are estimated from fitting those. But then you repeatedly
sample those 4 points but allowing some of the values to be repeated (and
so some ignored) and you fit that. The standard deviation of the results
of those fittings (for each parameter separately) is taken as an estimate
of the error for that parameter. The authors of this paper swear it's the
way it should be done. I don't know.
'jiggling':
No reference here because we made it up and I suspect it might well be
dubious. So here we repeatedly take samples of the data using normal
distributions around x and around y (for each point) with standard
deviation the estimates of the "noise" in x and y (respectively). So the
first sample would fit (x1+delta_x1, y1+delta_y1), (x2+delta_x2,
y2+delta_y2), etc., where the deltas are sampled as described. Then you
do the next sampling, etc. The error estimate is the standard deviation
of each parameter. I mean this just sounds dubious to me. And it's
pretty sensitive, we found. So I'd ignore this definitely.
Now there is (at least) one other subtlety here, for all three methods,
namely that the parameters that we are fitting are (quite often) not the
parameters you are (directly) intersted in. So you might be interested in
some value v = f(p) where p are the parameters of the fit and f is some
function. If you have an estimate p0 for p then I would say it's probably
ok (but not totally obvious) that you can take v0 = f(p0) as an estimate
of v. But with errors I don't really know whether there is an accepted
way to go from an estimate of the error of p to the error of v. You can
try a Taylor series expansion but it's just not clear to me how valid that
is. (It might be ok, though.)
I hope that helps (a bit!).
Wayne
On Tue, 17 Nov 2009, Paul Driscoll wrote:
> Hi,
>
> In the data fitting area of Analysis we are offered error handling options of
> 'covariance','bootstrap' and 'jiggling'. Is there any description anywhere of
> the relative merits or de-merits of these different options? Is any of them
> similar or identical to using 'Monte Carlo' generation of data sets within the
> uncertainties of the measured values (such is often used in some other
> implementations and is a method that I intrinsically understand)?
>
> Specifically, if I am trying to extract K_D for a ligand binding series what is
> the best Analysis-based way of assessing the confidence limits on the fitted
> values? (i.e. ideally I don't want to have to go to another program to sort this
> out.)
>
> [BTW, if you Google 'jiggling' you quickly run into material that can get one
> fired from one's job; even 'jiggling fitting' does not help and I decided not to
> go further.]
>
> Thanks in advance,
> Cheers,
> Paul
>
>
> --
> Paul C. Driscoll
> Honorary Professor of Structural Biology
> Structural and Molecular Biology
> University College London
> Gower Street
> London WC1E 6BT, UK
> Tel.: 44-20 7679 7035
> Mobile: 07876 777937
>
> -------------------------------------------------
> This mail sent through UCL Biochemistry's Webmail:
> http://webmail.biochem.ucl.ac.uk/ (unsecured)
> https://webmail.biochem.ucl.ac.uk/ (secured)
>
