Philipp,

> having read the archived mails on FDR multiple test correction in
> VBM I am still not sure about how to proceed with the preliminary
> results.
[...]
> At uncorrected p < 0.001 correlation maps show activations well
> balanced between focal/regional (not too excessive), also reasonable
> variation between the tests is found. Correction at FDR 0.05 tends
> to lead to rather excessive activations in most contrasts.

This last finding suggests that you have extensive regions with a
covariate effect.

> Using a more stringent q < 0.005 brings produces areas about
> comparable with p uncorr 0.001. However, with the FDR being adaptive
> this differs between contrasts: for rather weak results uncorrected
> 0.05 seems more appropriate.

It is very difficult to compare results across different multiple
testing methods.  0.05 uncorrected will always be the most sensitive
option, though with little specificity.  You've accurately noted that,
with FDR, the results even vary between contrasts, since different
contrasts can have differing number of voxels with large effects.

> Using the script FDRill the T-distribution is clearly shifted (up to
> about +2), so the question arises which inferences I can draw from
> such a model.

A shifted T-distribution tells you that you have either spatially
expansive effects or consistent but diffuse effects.  An expansive
effect would be observed by a huge blob; a diffuse effect by small
regions of significance scattered over the brain.

In either case, this 'sensitizes' FDR, since lots of significant
voxels detected means lots of false positives are OK (in proportion,
for a given q).  I realize this is unsatisfying, and this is a
direction in multiple testing research (controlling the maximal number
of false positives, instead of the proportion).


> The questions are:
> 1. Is FDR appropriate and how to I deal with the adaptiveness of
> this type
> of multiple test correction, i. e. is it appropriate to use
> different q
> values to different contrasts - which obviously contain results at
> 'different strength'?

The fact that you need to search over different q's indicates that
more work needs to be done on finding an acceptable multiple testing
false positive metric.  I don't have a good answer for this, but given
the existing technology, I would argue that different q's (selected
from a small set, say e.g. 0.05, 0.01, 0.005) are OK for different
contrasts as long as it is justified by a objective heuristic.

For example, a poor estimate (but an estimate none-the-less) of the
number of false positives is {#suprathreshold voxels}*q.  You could
start with q=0.05, compute estimated number of false positives, and if
that's too large, click to the next larger q.  I make no claims on the
optimality of this procedure, but as long as the set of possible q's
is small, it doesn't sound like a fishing expedition.

> 2. Can results - as earlier suggested - be presented i the way that
> uncorrected p=0.001 clusters are listed and such clusters are
> reported which
> show FDR lower than 0.05? (which would include the ones surviving
> lower
> thresholds?)

Again, uncorrected results can't be compared directly with (FWE or
FDR) corrected results.

> 3. T-distribution problem: I have not modelled total GM volume as
> covariate because we were also looking for regional (e. g. a whole
> lobe) effects and thought this would be over-corrected by such a COV?
> How can the conditions to use FDR be achieved?

I think this explains alot.  The lack of total GM volume adjustment
isn't so much an FDR issue as an interpretation issue.  By not
correcting by total GM volume you are discounting effects that could
be attributable to gross (diffuse) differences in brain volume.  If
you think there is something informative in temporal lobe volume's
contribution to global GM volume, I would compute a global GM volume
that excludes the temporal lobes and use that as a covariate.

Hope all of this helps!

-Tom

     -- Thomas Nichols --------------------   Department of Biostatistics
        http://www.sph.umich.edu/~nichols     University of Michigan
        [log in to unmask]                     1420 Washington Heights
     --------------------------------------   Ann Arbor, MI 48109-2029