Dear Kevin,

Just a few comments, in the hope that the statisticians will be able to
give you a proper answer in due course:

>1) I think I understand what inferences can be drawn from the voxel-level
>(amplitude), cluster-level, or set-level statistics spm99 reports.
>However, obviously in a given dataset, one level can be very significant
>and another not at all significant. Also it seems obvious that one should
>decide which one to believe a priori. Do you have a recommendation? If I
>review a manuscript that uses spm99, which level must be significant in
>your opinion? (Personally, the cluster analysis seems closest to the
>spatial resolution at which most people discuss their results, so I would
>usually think we should use them and ignore the voxel and set statistics.)

If corrected statistics are used, then one could report results without
deciding which level to use a priori.  Strictly speaking, if you are
looking at all three levels, then I suppose that an extra correction for
those three comparisons should be applied (or maybe 'two-point-something'
independent comparisons, since clearly the three are not independent), but
this will only be relevant for the most borderline levels of significance.

Most experiments are designed to ask whether there is activation in a
particular area or set of areas, rather than to ask 'is there any
significant difference in brain activity between my two conditions'.
Therefore the real choice is usually between cluster level and voxel level.
Since it is perfectly possible to have a very small, but highly significant
cluster of voxels (which might easily exceed the voxel level, fail to reach
significance at the cluster level), my primary concern would be to avoid
missing such clusters, so I would always look at the voxel level first.
However, I would also at least have a look at an spm with a lower
voxel-level threshold, so that I could see if there were any very large
clusters that didn't quite make it over the voxel-level threshold.

I guess if I did an experiment in which I specifically expected a weak
signal spread over a large area, then I might a priori decide that I was
going to look at the uncorrected cluster statistics for this
(pre-specified) area, but this is a situation that I have never faced to
date.

> 3a) About the change in corrected voxelwise p value: a previous email
>(<http://www.mailbase.ac.uk/lists/spm/2000-02/0036.html>http://www.mailbase.ac.u
>k/lists/spm/2000-02/0036.html) suggests that the only change from 96 to 99
>was that SPM99 no longer assumes that the variance is uniform across the
>image (or at least this is how I interpret the math-ese). However, isn't
>it also true that the results are now obtained from direct estimates of
>how often a T-image of specified smoothness would have any pixel of
>such-and-such peak value, rather than the pointwise conversion to a
>Z-image followed by inference based on how often a Z-image would reach the
>corresponding peak value? Are there any other changes in how significance
>is attributed in spm99?

Karl seems to have answered your last question in that very e mail.  He said:
"The only difference between SPM96  and SPM99 (in terms of estimation and
inference) is that the spatial smoothness estimator has been upgraded. This
upgrade accommodates non-stationary spatial correlations among the errors
when correcting for the search volume using tests based on peak height (but
not on those based on spatial extent, at this stage)."

>3b) The cited email calls the new (spm99) implementation an advance and
>says one can be more confident in its results. My read of this cautious
>description is:  any analysis that used spm96 can not be trusted due to a
>high false positive rate, at least any analysis that has "low" (!!) df and
>relies on the corrected voxelwise / amplitude p value. Is this correct?

This seems to me to be a bit unfair.  SPM has to make some assumptions to
get any statistics out at all.  Theoretical limitations made it necessary
to assume stationary smoothness in the implementation in SPM96.  For this
reason among others, people always smoothed their data before analysis,
tending to minimise non-uniformities of the actual degree of smoothness.
Under these conditions, SPM96 was reasonably reliable.  To say 'any
analysis that used SPM96 cannot be trusted' is surely putting it too
strongly.  One would only place a heavy reliance on highly significant
results (obtained from pre-smoothed data); in practice this is probably
only an issue for borderline results, such as those in your examples.

>3c) Why does the corrected clusterwise p value change from spm96 to spm99?
>Is it entirely due to the removal of the "nonstationary" assumption and
>the use of T rather than Z fields, as above, or is there an additional
>change in how the size-plus-amplitude p value is determined?  And which is
>correct, spm96 or spm99?

I think that we can be reasonably confident that in developing SPM99, the
authors haven't deliberately introduced an error that wasn't there in
SPM96!   As Karl has said in previous e mails, the recommendation must be
to use the latest version of SPM.  To describe a previous version as
'incorrect' because it made use of an simplifying assumption, explicitly
stated, seems unfair to me.  The calculated 'p' value is an estimate, and
one can improve on that estimate without rendering the first estimate
'incorrect' (even the values in SPM99 are still only estimates).

>3d) Did spm96 directly use the number of resels in computing corrected
>voxelwise significance, a la earlier versions of spm96?  And spm99 does
>not -- right?

It seems to me that you have to use resels to calculate corrected
significance.  Or at least, however you adjust your Bonferroni correction
to allow for the smoothness of the data, the adjustment can be thought of
as equivalent to using a certain number of resels.  However, the ratio of
voxels to resels need no longer be uniform across the whole image.

Best wishes,

Richard.

from: Dr Richard Perry,
Clinical Research Fellow, Wellcome Department of Cognitive Neurology,
Darwin Building, University College London, Gower Street, London WC1E 6BT.
Tel: 0171 504 2187;  e mail: [log in to unmask]
Pager: 04325 253 566.




%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%