Nicola,

> In my VBM analysis comparing two groups of subjects (patients
> vs. controls) I have some areas of the brain which are significant
> at voxel-level FWE-corrected P-values ranging from 0.005 to 0.087
> and whose location is highly consistent with my study hypotheses.

This is really the tyranny of "0.05".  FWE is an very stringent
measure of false positives, and 0.087 FWE isn't so much "no evidence
against the null", but rather as "weak evidence against the null while
controlling for chance of one or more false positive voxels".
Remember that a 0.05 FWE threshold means that, in the long run, only 1
out of 20 contrasts you look at will have *any* false positive
voxels. But I understand your concerns with publication.


> Nevertheless, since some of them are close (0.087) but not below the
> FWE-corrected P-value, I am not allowed to state in my report that I
> am going to discuss only the results surviving the FWE-correction
> and I am forced to go for the fall-back of FDR-correction. If I do
> this, I get a bunch of areas throughout the brain, some of them
> having an FDR-corrected P-value less than 0.05 but a FWE-corrected
> P-value of 0.998 which, I guess, are definitely not worth being
> reported.

Not necessarily.  FWE is a very stringent correction, and moreover,
the random field (or, more likely Bonferroni) method that gave you
that 0.998 P-value is probably conservative.

> But, if I decide to go for the FDR-correction, I think I
> am not allowed to choose which results are really significant and I
> need to report all of them even if the FWE-corrected P-value is
> "unbelievably" not significant. I hope I made myself clear enough
> about the problem I have been currently struggling with. Do you have
> any suggestions?

There's nothing magical about 0.05 FDR threshold.  While I think most
people would agree that they wouldn't like their expected proportion
of false discoveries (FDR) to be greater than 0.05 or 0.1, there's
nothing to stop you from using a more stringent FDR threshold, say
0.01 or 0.001.  If I get many many thousands of voxels detected I
often will use a FDR threshold smaller than 5% as I'm not comfortable
with the risk of of so many false positives in absolute terms.

You should of course worry about fishing for the right threshold.  If
I were a reviewer I'd be suspicious of a, say, 0.01394 FDR threshold;
I'd only consider a small handful of typical FDR levels.

I think that the "right" answer here is that we need to develop
methods that control quantities that are meaningful to investigators.
What would you like to do?  Control the chance of x or more false
positives at 0.05? (There's some work on this [1], but none that I
know that is applicable to imaging). Be 95% confident that no more
than 5% of the detected voxels are false positives?  (This has been
done for random fields [2] but isn't yet implemented in SPM; note that
this isn't "FDR", as a 5% FDR threshold only ensures that the long-run
average fraction of false discoveries is less than 5%).  Something
else?  I'd like to hear what would really make sense for you and other
users.

-Tom


    -- Thomas Nichols --------------------   Department of Biostatistics
       http://www.sph.umich.edu/~nichols     University of Michigan
       [log in to unmask]                     1420 Washington Heights
    --------------------------------------   Ann Arbor, MI 48109-2029



[1] Korn EL, Troendle JF, McShane LM and Simon R.  Controlling the
number of false discoveries: application to high-dimensional genomic
data.  Journal of Statistical Planning and Inference 124: 379-398, 2004.

[2] Perone Pacifico M, Genovese C, Verdienlli I. and Wasserman L.
False discovery control for random fields. Journal of the American
Statistical Association, 99:1002-1014, 2004