Hi Mike and all,

my take is basically mixing procedures like CDT 0.005 in SPM and AFNI clusterSIM on top simply doesn't control your type 1 FWER, no matter what you do. I'm CC Anders here who obviously knows better since he ran all the tests. I'm afraid there is no way around that thresholding results (beside switching the Bayes).

On the paper itself, resting state data are used to run fake task designs, so the stats results apply to task fMRI!

Dr Cyril Pernet,
Senior Academic Fellow, Neuroimaging Sciences
Centre for Clinical Brain Sciences (CCBS)

The University of Edinburgh
Chancellor's Building, Room GU426D
49 Little France Crescent
Edinburgh EH16 4SB
[log in to unmask]

From: Mike <[log in to unmask]>
Sent: 21 July 2016 19:48:02
To: [log in to unmask]; PERNET Cyril
Subject: Re: cluster failure article
Hi Cyril,

    Thanks for the link. It's really a nice interpretation about the article by Eklund et al. in PNAS, especially for those like me who is not an expert in statistics. Here I have two more questions.

    First, before the publication of that PNAS article, I have just finished writing a manuscript, in which we used a cluster-defining threshold (CDT) of 0.005 and 3dClustSim (0.005 seems wildly used in 3dClustSim) to calculate the threshold, which is 130 voxels. From your link and the PNAS paper, I understand that the p=0.01 and p=0.001 CDTs’ FWE with 3dClustSim are about 27% and 9%, respectively." My experiment is a 2(A and B)x2(X and Y) design. In other words, the regressors of interest contain AX, AY, BX, BY. We are interested in contrasts "(AX>AY)," "(BX>BY)," and "(AX>AY)>(BX>BY)," particularly the last one. Suppose "(AX>AY)" yields activation in brain regions Q and S, and "(AX>AY)>(BX>BY)" yields S and T. For each of the 3 contrasts, since we use a CDT of 0.005, its FWE with 3dClustSim should be about, say, 20%. If I perform a conjunction analysis of "(AX>AY)" and "(AX>AY)>(BX>BY)", i.e., to search for the overlap by directly using their group statistical map thresholded at 0.005 with k=130, and this conjunction analysis reveals overlap in brain region S, can I say that the error rate was more adequately controlled? The rationale is that the error rate for the identification of region S should be about 20%*20%=4%, which is <0.05.
    I'm just wondering whether, for people who already use 0.005 with 3dClustSim and are not willing to re-do all analyses, conjunction analysis could help.

    The second question is, since the inflation of FWE rate observed in the PNAS paper is from the simulation of resting-state data, can the conclusions (e.g., we need to stick on a stringent CDT, such as 0.001; voxel-wise thresholding is generally better than cluster-wise approach) be generalized to task-related fMRI data?

    Thanks in advance.