> PS If you found a reference suggesting that mulitple ROI analyses are exempt
> from the multiple comparison problem, I would be amused (in a dark way)

Dear all,

I'm not sure it will go so far as to justify multiple uncorrected ROI
analyses (or to amuse Karl...) but arguments against multiple
comparison correction have been made, including some highly cited
papers in respected journals, like:
  http://www.bmj.com/cgi/content/full/316/7139/1236

One counter-argument is that one only really needs to be concerned
with the false positive (or discovery) rate in aspects of independent
scientific interest, and that one shouldn't be penalised for combining
two or much such investigations in a single paper. For example, if one
were interested in two quite distinct brain regions (say in
morphometry, where manual delineation might involve substantial work),
and planned to report findings (whether significant or not) for both
of them, then it might seem peculiar that one would have to double the
p-values if one was able to perform delineations simultaneously and
submit a single large paper, whereas no correction would be called for
by reviewers if one instead did the work sequentially and submitted
two papers. A related argument is that if one simply reports all
findings, the reader can judge for themselves the multiplicity (based
on their own interpretation of the independence of the questions).

Of course, with VBM/SPM, all multiple voxels are certainly not of
independent scientific interest; one might try to argue that voxels in
some region(s) are of independent interest to those in other
region(s), but the complexity of this dependence and consequent
difficulty of interpretation for readers means that one cannot avoid
formal multiple comparison correction.

The case of a small number of ROIs in fMRI probably lies somewhere in
between these extremes. If one is looking for activation in any of the
ROIs, and it is the activation and not the specific location that
would be of interest (and especially if one would not report negative
results in the other ROIs) then multiple comparison correction would
be necessary. If one could argue that activations in region X and
region Y have different underlying bases and different
interpretations/implications and one planned a priori to report
findings in both, then I think  a case could be made for not having to
correct.

A related issue arises quite frequently with people investigating
positive and negative one-tailed t-contrasts. Arguably, if findings in
either direction could be considered interesting, the p-values should
be doubled to correct for the multiplicity (or one could use an
F-contrast), but this is rarely done. Again, the defence might be that
the two directions were of independent interest (e.g. activation and
deactivation with different causes/interpretations, or expected
atrophy vs unexpected (registration error?) in VBM), though this seems
not to be explicitly stated in papers which report both one-tailed
tests.

I hope that's of some interest!
All the best,
Ged