SnPM users,
The following is a reply to a reviewer who asked me about a methods
description in a manuscript using SnPM (an application paper, not a
methods paper). I thought the reply would be useful to all.
> What I should look for in the description? Mostly folks describe
> the preprocessing steps and then just say they used SnPM to generate
> P values.
Long answer first: The three most useful things in SnPM right now are
One Sample T test aka "MultiSub: 1 conditions, 1 scan per subject"
Two Sample T test aka "MultiGroup: 2 groups, 1 scan per subject"
Correlation aka "SingleSub: Single covariate of interest"
The other useful thing in SnPM is the option of using variance smoothing.
The three models are the workhorses of random effects models (as per
Holmes & Friston). Those alone (with out variance smoothing) give users
a means to overcome the conservative RFT results. Without variance
smoothing, all SnPM is doing (essentially) is giving a better corrected
thershold, which a user may just take back to SPM99 to create thresholded
maps.
However, with variance smoothing, you often can get even more sensitive
results (relative to SnPM w/ no var sm), especially with low-low df (<10).
The only other subtlety is how many permutation were used. Sometimes
there are only so many possible permutations (e.g. if you only have 7
subjects, there are only 2^7=128 possible ways of flipping the signs
on the subject's data. With correlation, with say 10 subjects, there
are 10! = 3,628,800 possible reorderings of the covariate values, and
you couldn't (and don't need to) consider all of them; you just do an
"approximate test" and use a fraction of the total number possible.
The number of permutations is important, since the corrected p-values
are multiples of 1/nPerm. If nPerm is less than 100 I would be
wary; I recommend at least 1,000 (if that many are possible from the
combinatorics), with 5,000 being better, and there being no point to
more than 10,000. (This may sound random, but there are actually some
stats behind it!)
SO, the short answer to your question is that I would check:
1. Is it clear which test they are using?
2. Have they specified whether they are using variance smoothing?
3. How many permutations are they using?
If they present the permutation distribution of the maximum statistic,
showing the threshold and the correctedly labeled maximum, even better.
-Tom
-- Thomas Nichols -------------------- Department of Biostatistics
http://www.sph.umich.edu/~nichols University of Michigan
[log in to unmask] 1420 Washington Heights
-------------------------------------- Ann Arbor, MI 48109-2029
|