 |
 |
> Okay, the "neuropsychological test battery" was a really bad > example. If your battery is large enough, and you correct for > multiple comparisons, it is very unlikely to find any true effects. > So I would not have corrected on F-test level (would have reported > both corrected and uncorrected p-values). > > Anyway, I had thought that I would have to take into account the > number of post-hoc tests (as well), but this seems to be wrong then. It isn't wrong, it is a problem that is not specific to neuroimaging. If you correct for the volume at each t-test, then you have the same problem you'd have if it was the neuropsychological test battery. Here, the F test does not solve the problem because the comparisons must be planned. You'd rather need a Bonferroni correction. The opinions on what to do in this situation vary. In genetic epidemiology, the tendency is to require large samples and (Bonferroni-corrected) significances much lower than 0.05 for credibility. In the political sciences, the view has been expressed that corrections are harmful because of the effect on type II error. Others like FDR approaches, which become increasingly attractive when the number of tests becomes high. > > > Back to the fMRI data. Imagine a purely within-subject 3x3-ANOVA, > which should be reasonable nowadays. E.g. something like "face" > (happy, sad, fearful), and "sex" (male, female, morph). Maybe I have > specific hypotheses, but maybe I do not (at least for some levels, > e.g. concerning "morph"). In the latter case, I would run F-tests > for "face", "sex" and the interaction. Imagine I get some clusters > surpassing an otherwise defined voxel-size threshold. What should I > do then? > > > Or should I run lots of t-tests right from the beginning? I would > already have to conduct 12 one-sided tests for "face" and "sex". And > to ensure that the results make sense, I would have to check all > the interactions as well. You could declare your t tests as explorative. You won't escape the problem by adopting one or the other approach to correction. To have an intuitive understanding of the inevitability of the problem, see it as a requirement on the resolution of your data. If you want high resolution (to figure out which of these many conditions is responsible for variance), you need more data; otherwise, you'll be looking at noise. Best wishes, Roberto Viviani Dept. of Psychiatry III University of Ulm, Germany