Print

Print

 
Dear all,


    I'd like to ask a question regarding appropriate post-hoc tests after 2*2 ANOVA whole-brain analysis.


    In our recent paper, 


        1). we performed a 2*2 ANOVA whole-brain analysis and identified several clusters in the 2*2 interaction (FWE-corrected at cluster level).


        2). we extracted average beta values from all activated voxels from each cluster identified with the interaction ("functional ROIs"). 


        3). we then compared these beta values between two conditions using paired-t tests in SPSS (e.g., A1B1 vs. A1B2: these are two interested conditions, and we want to confirm that the interaction was driven by the differences between them).


      According to the opinion of a reviewer (see below), the above "post-hoc tests" are not independent and can not be used. However, similar procedures can be observed in recently published papers (the bottom links provided several examples). Therefore,we are confused about what should be an appropriate approach to do the post-hoc tests after ANOVA whole-brain analysis (The reviewer kindly provided a solution for us, but it seems that this procedure makes more sense if we focused on ROI analyses at the very beginning).




Thanks in advance,
Chunliang








 Reviewer's comment: The authors define ROIs using the interaction between factor A and B and then run statistical tests on the data (post-hoc comparisions) extracted from those ROIs, specifically testing one half of the interaction (A1B1 > A1B2). These analyses are non-independent and should not be reported (see e.g. Kriegeskorte et al., 2009 Nature Neuroscience for further discussion of this issue).One alternative would be to use a Leave-One-Subject-Out (LOSO)approach: ROIs could be iteratively defined on n-1 subjects and then data extracted for the subject removed. This would ensure statistical independence between the contrast used to define ROIs and the tests performed on that data. Barring this or comparable analyses, the authors are limited in the extent to which they can make inferences about activation differences in the interaction contrast.
Relevant papers:http://scan.oxfordjournals.org/content/early/2013/10/26/scan.nst155.full.pdf+html


http://ajcn.nutrition.org/content/99/2/258.full


http://scan.oxfordjournals.org/content/8/1/65.full