 |
 |
Dear Natalie, We are addressing some reviewer concerns. My block design is ABACABAC. At > the subject first-level, I modeled each of my three conditions (3 separate > EVs), and specified two contrasts: B>A, and C>A with standard cluster > thresholding z-score of 2.3 and significance level of p=.05. For contrast > B>A (the EV for C was set at 0). Likewise, for contrast C>A (the EV for B > =0). > > At the higher-level, I used Flame 1+2, cluster Z>2.3, p<.05. > > The reviewer noted that two separate analyses were performed for B and C > and said this doubles chances of finding a result by chance. I have been > asked to either correct for repeat testing or incorporate both conditions > into the model. > > How can I correct for repeat testing? I understand I can’t use p=0.025 for > example as changing the p-value in this way would correspond to a > voxel-wise uncorrected value and does not relate to the final cluster-level > corrected p-value. > A Bonferroni correction can indeed be applied to (familywise) error corrected P-values; you simply need to change the "p<.05" above to "p<.025". I'm not exactly sure what the reviewer is getting at. They *might* be wondering if a common effect in B>A and C>A is due to a *decrease* in A; if you have any rest scans in your design you can try to look at the pure effect of A<0 to check this. Or, they might be after the usual multiplicity issue, that any time you look at multiple tests you increase your risk of false positives. I should say this second concern is a fairly knotty issue. What if you had only reported on B>A and then passed your data on to a friend who published C>A? Should you have corrected for the inference across the two papers? There's no hard and fast rule, but one line of reasoning goes like this: If you needed to look at all of a set of contrasts to answer a scientific question, then you should be correcting for multiplicity. If each contrast answers a distinct question that you interpret in isolation (and could, conceivably, be written up on it's own as a scientific work), *and* there aren't *too* many of them in total, then you can get away with out correction. There's no hard rule on this, and I won't try to define "too many", but here's an example of the former: Say you fit an FIR HRF, where you have, say, 10 EV's that model the HRF response at each of 10 lags. I don't think any reasonable person would say each of the 10 COPEs are answering a distinct question, and you'd have to deal with the multiplicity over the 10 tests. Hope this helps! -Tom -- __________________________________________________________ Thomas Nichols, PhD Principal Research Fellow, Head of Neuroimaging Statistics Department of Statistics & Warwick Manufacturing Group University of Warwick, Coventry CV4 7AL, United Kingdom Web: http://go.warwick.ac.uk/tenichols Email: [log in to unmask] Phone, Stats: +44 24761 51086, WMG: +44 24761 50752 Fax: +44 24 7652 4532