Say I conducted a single factor ANOVA with 4 levels and obtained a
significant F-test. I then would want to carry out 6 pairwise multiple
comparisons to identify the location(s) of the effect.
When exerting Type 1 error control, what is the convention regarding the
initial omnibus hypothesis test? Do I include the omnibus hypothesis test
and control over 7 hypotheses tests, or do I ignore the omnibus hypothesis
test and control over just 6 hypotheses tests?
My feeling is the omnibus hypothesis should be accommodated. However doing
so seems to create problems. For example, if I had organized my hypotheses
tests into two 3 hypotheses families, how should the omnibus hypothesis be
accommodated? It would be inappropriate to include the omnibus hypothesis
twice (i.e., in effect assess two 4 hypotheses families), but it seems
equally inappropriate to consider two hypotheses families: one with 4 and
one with 3 hypotheses (with the omnibus randomly allocated to a family!). I
suppose Type 1 error rate control could be done with two families each with
3.5 hypotheses!
This matter does not seem to addressed in any of the texts or papers I have
consulted - is it because I have the wrong end of the stick here?
Andrew Rutherford
|