Hi Tom,
Thank you for clarifying that. It makes
sense.
Is there any way to use both an ROI mask
image and the implicit zero masking? I noticed that, when spm_uc_FDR takes an
ROI mask image as variable Vm, it doesn’t do implicit masking even though
it does
Thanks,
-Satoru
From: SPM (Statistical
Parametric Mapping) [mailto:[log in to unmask]] On Behalf Of Thomas Nichols
Sent: Tuesday, February 24, 2009
3:46 PM
To: [log in to unmask]
Subject: Re: [SPM] wfu pickatlas
problem?
Hi Satoru,
Just a follow up, there's no need to edit spm_uc_FDR. The change
you made essentially 'locks on' implicit zero masking; instead, masking is
meant to be specified with the last argument.
For example, in spm_getSPM.m, the function is called as
u = spm_uc_FDR(u,df,STAT,n,VspmSv,0);
where the 0 argument is telling spm_uc_FDR to ignore zeros. See
the usage for the full details...
% Vm - Mask in 1 of 3 forms
% o Scalar, indicating implicit mask value in statistic image(s)
% o Vector of indicies of elements within mask
% o Mapped mask image
% (Ignored if Vs is a vector.)
If you edit the PickAtlas calls accordingly you can use the
factory-supplied spm_uc_FDR.m.
-Tom
On Tue, Feb 24, 2009 at 12:38 AM,
First, let me apologize for a rather harsh tone of my
language in my email below; it was meant to be an internal memo in our group
but somehow it got posted on the SPM mailing list unedited.
It turned out, the strange phenomenon described below is
caused by a mismatch between the number of in-mask voxels between spm_uc_FDR.m
and spm_P_FDR.m. You can correct this problem by editing a line in spm_uc_FDR.m
function. Alternatively you can edit two lines in wfu_spm_getSPM5.m function to
achieve the same result. The edited versions of both functions are
attached. Either of these changes should correct the problem.
Finally, being a statistician, I don’t think it’s
a good idea to relax the threshold until you see a blob. I know some people do
this, but changing the threshold after seeing the results doesn’t sound
like a good science.
-Satoru
From: SPM (Statistical Parametric
Mapping) [mailto:[log in to unmask]]
On Behalf Of Simon Jones
Sent: Monday, February 23, 2009
11:24 AM
To: [log in to unmask]
Subject: Re: [SPM] wfu pickatlas
problem?
Dear Satoru and Tom
I had a similar issue as Wei-chun Wang.
I have analyzed some data and using FDR 05 threshold
within an ROI in the PickAtlas and receive no suprathreshold clusters.
But subsequently using the standard SPM SVC clusters are detected.
Page 1. Standard SPM analysis using uncorrected threshold
p 001. Some coordinates appear to have an FDR significance < 05.
Page 2. Standard SPM using FDR corrected threshold
identifies the same coordinates.
Page 3. Apply SVC to page 2 on an ROI results makes the
FDR corrected values slightly more significant.
Page 4. Same data using PickAtlas with the ROI in 3 and
uncorrected threshold p 001. Similar results to 1.
Page 5. Same data using PickAtlas with the ROI and FDR
corrected threshold p 05 and no significant voxels are found.
I think SPM SVC applies threshold to data already passed
through to the results so the PickAtlas differs? If I pick a higher FDR
threshold than 0.05 such as 0.1 then the significant voxels are reported by the
PickAtlas with values like 0.034 etc. I attach a graph of the statistics
similar to Satoru’s. The blue SPM FDR 05 crosses once, the red
PickAtlas 0.05 do not cross (so no clusters found), the green PickAtlas 0.1
threshold actually crosses the red wiggly line 3 times ( the point at 114 is
off the scale shown) so this time it finds clusters. What conclusions can
be made? You can see in page 4 that some of the main blob is cut by the
ROI. I am not trying to tweak thresholds.
Thank you for your help
Simon
From: SPM (Statistical Parametric
Mapping) [mailto:[log in to unmask]]
On Behalf Of Thomas Nichols
Sent: 23 February 2009 13:23
To: [log in to unmask]
Subject: Re: [SPM] wfu pickatlas
problem?
Dear
Joe et al,
In
fact there's no problem with the plot of P-values starting above the line.
For a level-alpha FDR procedure, all you need is one or more P-values
that fall below the slope-alpha line, and the largest such P-value defines the
FDR threshold. In statistics lingo, this Benjamini & Hochberg FDR
method is a "step-up" test, which essentially works from worst
P-value and steps 'up' until it finds a significant P-value and
rejects that null and all others with smaller P-values; this is in contrast to
a step-down test which starts from the most significant P-value, and and stops
at the first non-significant P-value. (I'm Satoru remembers this, as we
co-wrote a paper explaining the difference [1] :)
Anyway,
I can't help explain Wei-chun's problem, but wanted to clarify this issue.
-Tom
[1] Thomas
Nichols and
Controlling
the familywise error rate in functional neuroimaging: a comparative review
Statistical
Methods in Medical Research 2003; 12: 419-446
On
Tue, Feb 10, 2009 at 9:52 PM, Joseph Maldjian <[log in to unmask]>
wrote:
Wei-Chung,
Below is the response from
dilemma...
So, Ben gave me a list of t-scores for this ROI, and I was able to
figure out why this person got strange results. As I said yesterday, the
FDR threshold is determined by plotting uncorrected p-values from the
smallest to the largest. Then you draw a straight line corresponding the
FDR value of your choice (0.05, 0.072, etc), and wherever the p-value
curve crosses this straight line is the FDR threshold. See the attached
figure for an illustration of this.
Now, in order for the FDR threshold to exist, the p-value curve should
start below the straight line, and crosses the straight line as it
increases. Unfortunately, this person's p-value curve doesn't start
below the straight line, so theoretically the FDR threshold is
undefined, and nothing survives the FDR correction. But this person
played with the FDR threshold value to cause a double-crossing (see
yellow line). This, in no way, makes the FDR-corrected p-values valid;
this FDR-threshold "fooled" (hopefully unintentionally) spm_P_FDR.m
and
made it produce invalid p-values. I guess whoever wrote spm_P_FDR.m
wasn't prepared for a scenario like this.
So, this is the answer to this conundrum. Nothing really survived; but
this person tweaked the threshold and produced the results which are
invalid and misleading. So, I guess you have to call a spade a spade.
Nothing survived at FDR<0.05, and that's the end of the story. It has
nothing to do with PickAtlas.
-Satoru
------------------------------------------------------------------------
Joseph A. Maldjian, MD
Professor and Chief of Neuroradiology
Director Advanced Neuroscience Imaging Research Core (ANSIR)
Office:336 716-2815
fax: 336 716-2870
email: [log in to unmask]
website: www.fmri.wfubmc.edu/maldjian.htm
------------------------------------------------------------------------
-
-----Original Message-----
From: SPM (Statistical Parametric Mapping) [mailto:[log in to unmask]]
On Behalf Of Wei-chun Wang
Sent: Monday, February 09, 2009 7:38 PM
To: [log in to unmask]
Subject: [SPM] wfu pickatlas problem?
I'm trying to use pickatlas with SPM and I'm running into a problem. I
have
a ROI that appears to be just under the threshold for significant after
small volume correction (.049 FDR). However, when I use pickatlas and
.05
FDR as my threshold, my activation does not reach threshold. HOWEVER, if
I
use .072 FDR my activations show up, and in the table the corrected p
value
is properly listed as .05
To sum, my problem is that the threshold has to be relaxed by ~.022 in
order
for the actual threshold (.05) to be seen by SPM.
I've included 3 screen shots that illustrate my problem:
"uncorrected" shows what happens if i set an uncorrected p-value and
then
use SPM's SVC function. you can see that the FDR corrected is .05
"pickatlas_05" shows what happens if i use pickatlas and set the
p-value
to
.05 FDR. no activations reach the threshold
and finally, "pickatlas_072" shows what happens if i use pickatlas
and
set
the p-value to .072 FDR. at this point it appears most of the
activations
that reach the .05 threshold appear.
This isn't a rounding error. The activations do not show up if I try
.051 or
.06, only at .072. Also matlab shows that the FDR corrected p-value is
~.049. I was wondering if this is a bug in the software and if there is
a
workaround? Or is it a problem in my data? I was hoping to use this
program
in order to produce accurate activation maps of just the small volume
corrected ROI. So alternatively, if anyone can recommend a separate
method
for doing so I would be quite appreciative.
Wei-chun Wang
--
____________________________________________
Thomas Nichols, PhD
Director, Modelling & Genetics
GlaxoSmithKline Clinical Imaging Centre
Senior Research Fellow
--
____________________________________________
Thomas Nichols, PhD
Director, Modelling & Genetics
GlaxoSmithKline Clinical Imaging Centre
Senior Research Fellow