On Fri, Oct 29, 2010 at 1:53 AM, Sherif Karama <[log in to unmask]> wrote:
> I agree with almost everything you wrote but I do have a comment.
>
> In a situation where I am expecting, with a very high degree of probability,
> activation of the amygdala (for example) and yet expect (although with
> lesser conviction) activations in many regions throughout the brain, the
> situation becomes rapidly complex.
>
> If one is looking only at the amygdala, one would be justified in using a
> small volume correction perhaps. But if one is looking at the whole brain
> including the amygdala, then it can perhaps be argued that whole brain
> corrections are needed. However, this last correction would not take into
> account the increased expectancy of amygdala activation. So an alternative
> may be to use modulated/different thresholds which would be likely viewed as
> very unelegant. Although somewhat of a Bayesian approach, here again one
> would be faced with quantifying regional expectancy (which can be
> very tricky business). It is for such reasons that I do consider findings
> from uncorrected thresholds sometimes meaningful when well justified. Here
> I am thinking of 0.001 or something like this which provides a certain
> degree of protection against false positives but also allowing for weak but
> real signals to emerge. Perhaps it's this kind of thinking that has led SPM
> creators to use a 0.001 threshold as default when one presses on
> uncorrected?
>
> Am any of this making sense to you?
>
I understand your problem but I don't think using uncorrected
thresholds are really the solution to it. For the specific example you
give I think doing small volume correction for the amygdala and then
normal FWE correction for the rest of the brain is a valid and elegant
enough solution. If you have varying degrees of prior confidence that
would indeed require a Bayesian approach, but I don't think many
people can really quantify their degree of prior belief for different
areas, unless it is done with some kind empirical Bayesian
formulation.
Statistics is not about the truth but it is a way of decision making
under uncertainty. And the optimal way to make such decisions depends
on what degree of error of each type we are willing to tolerate. I
would argue that although in the short term one is eager to publish a
paper with some significant finding, using very liberal thresholds is
damaging in the long term. You will eventually have to reconcile your
findings with the previous literature which might be very difficult if
this literature is full of false positives. Also building any theories
is made difficult by the high level of 'noise'. Eventually not being
conservative enough can ruin the credibility of the whole field.
The problem with uncorrected thresholds is that you can't even
immediately quantify your false positive rate because it depends on
things like the number of voxels and degree of smoothing. I think the
reason the uncorrected option is there is because some people use it
for display and for diagnostics. Also there are many ways to define
significance and if one was only allowed to see an image after
specifying exactly the small volume or the cluster-level threshold
it'd make the user interface more complicated.
Try adding random regressors to your design and testing for them with
uncorrected threshold to convince yourself that there is a problem
there. With that said it's all a matter of community standard. Ffor
instance a purist would also do a Bonferroni correction between all
the tests reported in a paper or even between all the different
variants of the analysis attempted. But I don't know many people who
do it ;-)
Best,
Vladimir
>
> On Thu, Oct 28, 2010 at 5:37 PM, Vladimir Litvak <[log in to unmask]>
> wrote:
>>
>> Just to add something to my previous answer, you can look up in the
>> 'cluster-level' part of the table what is the size of the smallest
>> significant cluster and then press 'Results' again and use that number
>> as your extent threshold. Then you'll get a MIP image with just the
>> significant clusters which is what you want.
>>
>> Vladimir
>>
>> On Thu, Oct 28, 2010 at 3:51 PM, Vladimir Litvak
>> <[log in to unmask]> wrote:
>> > Dear Sun,
>> >
>> > On Thu, Oct 28, 2010 at 3:32 PM, Sun Delin <[log in to unmask]> wrote:
>> >> Dear Vladimir,
>> >>
>> >> Thank you so much for the detailed reply. Could I conclude your
>> >> replies as follows?
>> >> 1. Try to do correction for multiple comparisons to avoid false
>> >> positive.
>> >> 2. If there is no hypothesis IN ADVANCE, SPM is better than SPSS
>> >> because the former can provide a significant map with both temporal and
>> >> spatial information.
>> >> 3. Use small time window of interest to do analysis.
>> >
>> > This is all correct.
>> >
>> >
>> >> 4. Cluster-level inference is welcome, so large extent threshold is
>> >> good.
>> >>
>> >
>> > You don't need to put any extent threshold to do cluster-level
>> > inference. What you should do is present the results uncorrected, lets
>> > say at 0.05. Then press 'whole brain' to get the stats table and look
>> > under where it says 'cluster-level'. You will see a column with title
>> > 'p FWE-corr' (third column from the left of the table). This is the
>> > column you should look at and if there is something below p = 0.05
>> > there you can report it saying that it was significant FWE-corrected
>> > at the cluster level. You can use higher extent threshold if you get
>> > many small clusters that you want to get rid of.
>> >
>> >> However, I would still like to ask more clearly
>> >> 1. If there is no significance left (I am often unlucky to meet such
>> >> results) after correction for multiple comparisons (FWE or FDR), could I use
>> >> uncorrected p value (p < 0.05) with large extent threshold such as k > 400?
>> >> Because it seems impossible that more than 400 adjacent voxels are all false
>> >> positive. If you are the reviewer, could you accept that result?
>> >
>> > No. You can't do it like that because although it is improbable you
>> > can't put a number on how improbable it is. What you should do is look
>> > in the stats table as I explained above.
>> >
>> >> 2. You said that it is "absolutely statistically invalid thing to do is
>> >> to find an uncorrected effect in SPM and then go and
>> >> test the same channel and time window in SPSS." However, I found that
>> >> if the uncorrected effect (e.g. p < 0.05 uncorrected, k > 400) appeared at
>> >> some sites in SPM, SPSS analysis involving the same channel and time window
>> >> would show a more significant result. Because most ERP researchers now
>> >> accept the results by SPSS, is it a way to use SPM as a guide to show the
>> >> possible significant ROI (temporally and spatially) and use SPSS to get the
>> >> statistical significance?
>> >
>> > No that's exactly the thing that is wrong. You can only use SPSS if
>> > you have an a-priori hypothesis. As I explained you will get more
>> > significant results in SPSS than in SPM because SPSS assumes
>> > (incorrectly in your case) that you are only doing a single point test
>> > and it doesn't know about all the other points you tried to test in
>> > SPM whereas SPM does know about them and corrects for this.
>> >
>> >> 3. If the small time window of interest is more sensitive, could I use
>> >> several consecutive small time window (e.g. 50 ms) of interest to analysis
>> >> long component such as LPC (I know some researchers use consecutive time
>> >> window to analysis LPC component by SPSS) or as an exploring tool to
>> >> investigate the possible significant result on dataset without hypothesis IN
>> >> ADVANCE?
>> >
>> > If the windows are consecutive (i.e. there are no gaps between them)
>> > then you should just take one long window. If there are gaps you can
>> > use a mask image that will mask those gaps out and SPM will
>> > automatically account for the multiple windows.
>> >
>> >> 4. Because of the head shape and some other reasons, the 2D projection
>> >> map of each individual' sensors on scalp is some different from the standard
>> >> template provided by SPM. Is it correct to put each subjects' images based
>> >> on their own 2D sensors' map into the GLM model for specification, or use
>> >> images based on standard 2D sensors' map instead? I have tested both ways
>> >> and found that the previous method may lead to some stripe like significance
>> >> at the border of mask. I do no know why.
>> >
>> > Both ways are possible. You can either mask out the borders if you
>> > know there is a problem there or use standard locations for all
>> > subjects.
>> >
>> > Best,
>> >
>> > Vladimir
>> >
>> >
>> >>
>> >> Sorry for asking some weak questions, however, I really like the
>> >> EEG/MEG module of SPM8.
>> >>
>> >> Bests,
>> >> Sun Delin
>> >>
>> >>
>> >
>
>
|