Hi Anderson and Jun
This evolved into an interesting discussion!
Just as an additional note: When it comes to ANOVA, I was always
taught to look for the "deepest" effect -- in the example of Jun's,
that would be the interaction effect. Similarly to what Anderson
wrote: "We only look into the main effects after ruling out an
interaction effect. ". Recently, I came across this blogpost
(http://deevybee.blogspot.de/2013/06/interpreting-unexpected-significant.html),
which basically says that it is much more likely to find a significant
result for the interactions than for the main effects. This made me
caution about interaction effects in the brain imaging (with multiple
comparison problem in mind) without showing main effects (whole brain
within or between subjects contrasts).
Thanks for the contribution to my initial question!!
Best regards, Blazej
On 20 April 2017 at 02:33, Anderson M. Winkler <[log in to unmask]> wrote:
> Hi Jun,
>
> There is nothing in the word "ANOVA", even considering the interactions,
> that changes the fact that these are just up 4 EVs that ideally are all
> orthogonal to each other. Justification for doing a two-step procedure, or
> for using a more liberal significance level, comes more from semantics. In
> my view, one should correct across all of them.
>
> Of course, classical texbooks cannot even bring this up -- trying to correct
> in the parametric case if the EVs aren't orthogonal would be cruel, and even
> with permutations it may not always be obvious, e.g., permutations need to
> be synchronised, and for certain contrasts (not the simple 2x2), some
> permutations don't change anything.
>
> All the best,
>
> Anderson
>
>
> On 19 April 2017 at 20:45, Jun Miyata <[log in to unmask]> wrote:
>>
>> Dear Anderson,
>>
>> Thank you for your feedback. I’m still confused about it, because
>> conventionally no correction is required over main effects and interaction
>> in the case of 2x2 ANOVA, that is, the statistical threshold for each is not
>> p<0.017=0.05/3 but p<0.05. How do you think about it?
>> many thanks in advance,
>>
>> Jun
>>
>>
>> > 差出人: "Anderson M. Winkler" <[log in to unmask]>
>> > 件名: Re: FDR and FWE TFCE correction for multiple images
>> > 日付: 2017年4月19日 11:10:56 JST
>> >
>> >
>> > Hi Jun,
>> >
>> > We only look into the main effects after ruling out an interaction
>> > effect. This would open the possibility of doing the correction in two
>> > steps, first correcting for the two interactions (contrasts 5 and 6), and
>> > then the four remaining ones. However, interpretation into main and
>> > interaction are is simply some semantic meaning given to a set of what
>> > otherwise are just numbers. I don't feel it is appropriate to separate in
>> > this case, and would correct across all six in a single step.
>> >
>> > All the best,
>> >
>> > Anderson
>> >
>> >
>> >> On 18 April 2017 at 21:09, Jun Miyata <[log in to unmask]>
>> >> wrote:
>> >> Dear Anderson and Blazej,
>> >>
>> >> Sorry for interrupting. I also have a question re control over the
>> >> number of contrasts. Suppose I want to test a 2 x 2 ANOVA design (2
>> >> diagnostic groups, 2 conditions). The design matrix would be as follows:
>> >>
>> >> Contrast 1: Healthy - Patient
>> >> Contrast 2: Patient - Healthy
>> >> Contrast 3: Condition A - Condition B
>> >> Contrast 4: Condition B - Condition A
>> >> Contrast 5: Diagnosis x Condition Interaction
>> >> Contrast 6: - (Contrast 5)
>> >>
>> >> In this case, I agree with controlling over 1 and 2, 3 and 4, 5 and 6.
>> >> However, I’m not sure if it’s necessary to control over the main effect of
>> >> diagnosis (1 and 2), main effect of condition (3 and 4), and interaction (5
>> >> and 6). I appreciate your opinion.
>> >>
>> >> Thanks in advance,
>> >>
>> >> Jun
>> >>
>> >> > 差出人: "Anderson M. Winkler" <[log in to unmask]>
>> >> > 件名: Re: FDR and FWE TFCE correction for multiple images
>> >> > 日付: 2017年4月18日 21:02:07 JST
>> >> >
>> >> >
>> >> > Hi Blazej,
>> >> >
>> >> > In the example of the paper, we treat each stimulation (face, hand,
>> >> > foot) as a different "modality" (even though it's all FMRI), and in PALM
>> >> > usage, these are entered each with an option "-i", and we'd use the option
>> >> > "-corrmod" to correct across them.
>> >> >
>> >> > The correction for multiple contrasts refers to the contrasts
>> >> > associated with a particular design for a given modality, and is enabled
>> >> > with the option "-corrcon".
>> >> >
>> >> > To make it more concrete: suppose in the 3 stimuli we test, for each,
>> >> > positive and negative effects across subjects. Then we have overall 3x2=6
>> >> > possible tests. The -corrmod alone would correct across the 3 stimuli, but
>> >> > not across the 2 contrasts. The -corrcon alone would correct across the 2
>> >> > contrasts, but not across the 3 stimuli. Using -corrmod together with
>> >> > -corrcon would correct across all the 6 things.
>> >> >
>> >> > Hope this helps.
>> >> >
>> >> > All the best,
>> >> >
>> >> > Anderson
>> >> >
>> >> > On 17 April 2017 at 12:56, Blazej Baczkowski
>> >> > <[log in to unmask]> wrote:
>> >> > Hi Anderson
>> >> >
>> >> > Thanks a lot ― PALM is very cool.
>> >> >
>> >> > There is one puzzling issue in my mind when it comes to multiple
>> >> > testing problem. Perhaps, you could help me to understand. In your paper
>> >> > (Winkler et al., 2016; Non-parametrc Combination…) you gave an example of a
>> >> > pain study. I wonder whether one needs to correct for multiple contrasts
>> >> > when testing the effects of each stimulation (face, hand, foot) separately?
>> >> > In other words, if an experiment consists of more than one condition, does
>> >> > one need to correct for multiple contrasts when examining the effects of
>> >> > each condition?
>> >> >
>> >> > Many thanks in advance,
>> >> > Blazej
>> >> >
>> >> > On 14 Apr 2017, at 19:14, Anderson M. Winkler
>> >> > <[log in to unmask]> wrote:
>> >> >
>> >> >> Hi Blazej,
>> >> >>
>> >> >> The simplest is probably assemble 4 separate one-sample t-tests, one
>> >> >> for each of the 4 levels. Each level act as a "modality". The call to PALM
>> >> >> would be something as this:
>> >> >>
>> >> >> palm -i level1.nii -i level2.nii -i level3.nii -i level4.nii -o
>> >> >> myresults -n 2000 -approx tail -logp -corrmod -corrcon -ise
>> >> >>
>> >> >> For a one-sample t-test, there is no need to specify design and
>> >> >> contrasts: these are created automatically if none is given. The option
>> >> >> -corrcon will correct for positive and negative contrasts, -corrmod will
>> >> >> correct for the 4 "modalities" (here are levels), the -ise will do
>> >> >> sign-flippings.
>> >> >>
>> >> >> Hope this helps!
>> >> >>
>> >> >> All the best,
>> >> >>
>> >> >> Anderson
>> >> >>
>> >> >>
>> >> >>
>> >> >> On 13 April 2017 at 16:11, Blazej Baczkowski
>> >> >> <[log in to unmask]> wrote:
>> >> >> Hi
>> >> >>
>> >> >> Thanks for the hint. I am not exactly sure, though, how to run the
>> >> >> analysis with PALM properly.
>> >> >>
>> >> >> My design is a form of repated measures one-way ANOVA, but at the
>> >> >> moment I am not interested in differences among levels, but the group mean
>> >> >> of each level. Therefore, I am not sure how to specify the design matrix. I
>> >> >> would like to follow the “recipe” on the randomise website
>> >> >> (https://fsl.fmrib.ox.ac.uk/fsl/fslwiki/Randomise/UserGuide#Repeated_measures_ANOVA)
>> >> >> 1 factor 4 levels, but I am not sure how I can get the mean for each level
>> >> >> with the contrast mentioned on the website.
>> >> >> Ultimately, I would like to have one image for group mean of each
>> >> >> level. Is there another way to run several one sample t tests? Or I should
>> >> >> treat my data as if they were seperate “modalities” in PALM?
>> >> >>
>> >> >> Do I understand correctly that once the design matrix is specified,
>> >> >> all I need to do to correct over multiple contrasts is to add the option
>> >> >> "-corrcon”?
>> >> >>
>> >> >> I would appreciate any feedback on this!
>> >> >> Best, Blazej
>> >> >>
>> >> >>
>> >> >> On 13 Apr 2017, at 12:47, Matthew Webster
>> >> >> <[log in to unmask]> wrote:
>> >> >>
>> >> >> > Hello,
>> >> >> > If you want to properly correct over multiple contrasts,
>> >> >> > then it’s probably best to use PALM (
>> >> >> > https://fsl.fmrib.ox.ac.uk/fsl/fslwiki/PALM ) instead of randomise.
>> >> >> >
>> >> >> > Hope this helps,
>> >> >> > Kind Regards
>> >> >> > Matthew
>> >> >> >
>> >> >> >> On 13 Apr 2017, at 11:20, Blazej Baczkowski
>> >> >> >> <[log in to unmask]> wrote:
>> >> >> >>
>> >> >> >> Hello everyone,
>> >> >> >>
>> >> >> >> I would like to find a single common threshold for multiple
>> >> >> >> statistical images (one sample t test, output from randomise) when
>> >> >> >> correcting for multiple comparisons. I would be happy to hear your opinion
>> >> >> >> on how to do it best. At the moment I am considering two options, but maybe
>> >> >> >> there are better ones.
>> >> >> >>
>> >> >> >> 1) FDR
>> >> >> >> Is it possible for the FSl fdr command to find a threshold when
>> >> >> >> the nifti image contains several volumes (concatenated *vox_p_tstat* maps
>> >> >> >> from randomise)? When testing, it works, but I would like to confirm that
>> >> >> >> the threshold I receive is indeed computed using all p values from several
>> >> >> >> volumes in the nifti.
>> >> >> >>
>> >> >> >> Would it also make sense to use uncorrected TFCE image
>> >> >> >> (*_tfce_p_tstat*) to preserve some spatial contingency among voxels?
>> >> >> >>
>> >> >> >> 2) TFCE and max value across images
>> >> >> >> I was wondering whether one could modify the randomise command
>> >> >> >> such that TFCE p values are corrected based on the maximum TFCE value across
>> >> >> >> all voxels and all images in each permutation providing a common
>> >> >> >> distribution of max values for all images?
>> >> >> >>
>> >> >> >> Many thanks in advance!!
>> >> >> >> Best, Blazej
>> >> >>
>> >> >
>
>
|