p-values for any single ANOVA are interdependent due to common
denominator
-------- Original Message --------
Subject: [SPAM] Multiple Comparisons revisited
From: Kim Pearce <[log in to unmask]>
Date: Fri, November 29, 2013 7:52 am
To: [log in to unmask]
Many thanks go to Robert Newcombe, Roger Newson, [log in to unmask]
and Neal Alexander who replied to my mail re. multiple comparisons.
If I may, I would like to ask a further question.
Suppose I am conducting a repeated measures analysis. Hypothetically I
have one within subjects factor (with 5 levels) and one between subjects
factor (with 3 levels) - say there are n different subjects in each
level of the between subjects factor. When I run the analysis I get a
significant interaction and so I decide to conduct 'simple main effects'
tests. I aim to establish if there is a difference between the 3 levels
of the between subjects factor at each level of the within subjects
factor. I realise that there are many ways to do this, but I decide to
do five 1-way ANOVAs...one for each level of the within subjects factor.
As we get 5 p-values I can apply, for example, the traditional
Bonferroni correction (i.e. comparing each p value to 0.05/5). Let's
assume that the results of the 5 ANOVAs are significant at the table
wide 0.05 level. Say if I want to subsequently look at Bonferroni post
hoc tests for each of these 5 ANOVAs to establish, for each within
subjects factor level, which pairs of the between subjects factor levels
are different. As there are 3 between subjects factor levels there will
be 3 pairs of levels which are compared (1 vs 3,1 vs 2,2 vs 3) for each
within subjects factor level. SPSS has an option to do these post hoc
tests but in one SPSS text I have read it implies that since we have
used 0.05/5=0.01 when looking at the significance of the 5 ANOVAs, we
should also choose a 'significance level' of 0.01 in the 'Post Hoc'
dialogue box (Analyze>Compare Means>One Way ANOVA) thus, for the
Bonferroni post hoc tests for each 1-way ANOVA, each comparison is
tested at the alpha level for the ANOVA divided by the number of
comparisons; for our example, 0.01/3....... I'd appreciate your views on
this.
Many thanks again for your advice on his matter ,
Kind Regards,
Kim
Dr Kim Pearce PhD, CStat
Senior Statistician
Haematological Sciences
Institute of Cellular Medicine
William Leech Building
Medical School
Newcastle University
Framlington Place
Newcastle upon Tyne
NE2 4HH
Tel: (0044) (0)191 282 0451
You may leave the list at any time by sending the command
SIGNOFF allstat
to [log in to unmask], leaving the subject line blank.
You may leave the list at any time by sending the command
SIGNOFF allstat
to [log in to unmask], leaving the subject line blank.
|