Dear Natalie,We are addressing some reviewer concerns. My block design is ABACABAC. At the subject first-level, I modeled each of my three conditions (3 separate EVs), and specified two contrasts: B>A, and C>A with standard cluster thresholding z-score of 2.3 and significance level of p=.05. For contrast B>A (the EV for C was set at 0). Likewise, for contrast C>A (the EV for B =0).
At the higher-level, I used Flame 1+2, cluster Z>2.3, p<.05.
The reviewer noted that two separate analyses were performed for B and C and said this doubles chances of finding a result by chance. I have been asked to either correct for repeat testing or incorporate both conditions into the model.
How can I correct for repeat testing? I understand I can’t use p=0.025 for example as changing the p-value in this way would correspond to a voxel-wise uncorrected value and does not relate to the final cluster-level corrected p-value.
A Bonferroni correction can indeed be applied to (familywise) error corrected P-values; you simply need to change the "p<.05" above to "p<.025".I'm not exactly sure what the reviewer is getting at. They *might* be wondering if a common effect in B>A and C>A is due to a *decrease* in A; if you have any rest scans in your design you can try to look at the pure effect of A<0 to check this. Or, they might be after the usual multiplicity issue, that any time you look at multiple tests you increase your risk of false positives.I should say this second concern is a fairly knotty issue. What if you had only reported on B>A and then passed your data on to a friend who published C>A? Should you have corrected for the inference across the two papers? There's no hard and fast rule, but one line of reasoning goes like this: If you needed to look at all of a set of contrasts to answer a scientific question, then you should be correcting for multiplicity. If each contrast answers a distinct question that you interpret in isolation (and could, conceivably, be written up on it's own as a scientific work), *and* there aren't *too* many of them in total, then you can get away with out correction.There's no hard rule on this, and I won't try to define "too many", but here's an example of the former: Say you fit an FIR HRF, where you have, say, 10 EV's that model the HRF response at each of 10 lags. I don't think any reasonable person would say each of the 10 COPEs are answering a distinct question, and you'd have to deal with the multiplicity over the 10 tests.Hope this helps!-Tom--__________________________________________________________
Thomas Nichols, PhD
Principal Research Fellow, Head of Neuroimaging Statistics
Department of Statistics & Warwick Manufacturing Group
University of Warwick, Coventry CV4 7AL, United KingdomWeb: http://go.warwick.ac.uk/tenichols
Email: [log in to unmask]
Phone, Stats: +44 24761 51086, WMG: +44 24761 50752
Fax: +44 24 7652 4532