also note that F-tests have a spherical or elliptical rejection region while individual t-tests evaluate rectangular rejection regions. This leads to differences in the rejection regions: points outside an ellipse may be detected by the F-test but, if they fall within a box, the H0 may not be rejected by the t-test. See
http://www.amstat.org/publications/jse/v16n3/martin.html (Tom Nichols alerted me once to Fig. 2).
It is a bit of a different plot than your example (because you are wondering about the discrepancy of positive detections) but it may be worthwhile to mention.
Hi Lisa,
Oops, my bad, it's Hayter, not Hayer: Hayter AAJ. The maximum familywise error rate of Fisher’s least significant difference test. J Am Stat Assoc. 1986 Dec;81(396):1000–4.
http://www.jstor.org/stable/2289074
However you don't have to bother much with it -- it refers to differences more in the opposite direction, i.e., excess false positives as opposed to not finding results in the t-test, and also it shows that with 3 groups it's fine. The fact that you are using cluster-level inference already explains the issue: there is no guarantee that the F-test will match the t-tests even in the 3-group case.
Which to choose then? I would go for the one that has higher specificity: the t-tests. I would leave the F-test results aside.
All the best,
Anderson
On 2 December 2016 at 12:28, Lisa Kramarenko
<[log in to unmask]> wrote:
Hi Anderson,
thanks for your response! Indeed, I am comparing three groups and I am using easythresh (after flameo) which performs multiple comparison correction at the cluster level!
Unfortunately I can't find the paper you're referring to, could you also tell me its name? Or maybe you can really briefly summarize what the reason for this discrepancy is?
And I have another question: would you mind giving a hint about how to interpret the not-matching results? Is it a difference to be reported or rather not?
Thanks so much for your help!