 |
 |
How Do You Control Type 1 Error Inflation when Comparing Independent Groups on Many Dependent Variables? Brief Background A research project requires 3 independent groups to be compared on 33 neuropsychological test score DVs. As interest focuses on which of the 33 DVs the groups significantly differ, multivariate or global null hypothesis test approaches merely postpone the inevitable consideration of the 3 groups compared on each DV. Consequently, 33 ANOVAs are required. Subsequently, covariates are employed and so 33 ANCOVAs need to be applied. Finally, 33 multiple regressions must be applied to examine if a small set of quantitative variables predict any of the neuropsychological test scores. Query 1 Is there a convention for controlling Type 1 error rate in these circumstances? Query 1 If not, how do you deal with the type 1 error inflation when so many tests are conducted? Apart from the very low power of a conventional Bonferoni adjustment in such circumstances, there are difficulties identifying the number of independent statistical tests applied - the number upon which the Bonferoni adjustment should be based. A principal components analysis of the 33 DVs could suggest how many of the 33 ANOVAs were independent . For example, if there were only 10 principal components underlying the 33 DVs would it be correct to say there were only 10 independent ANOVAs?. Even if so, how could you determine the independence of these 10 ANOVAs from the 33 ANCOVAs? Ultimately, how could you determine how many independent statistical tests existed amongst the 33 ANOVAs, 33 ANCOVAs and 33 multiple regressions? Trying to determine the number of independent tests in such a situation seems almost impossible. But if this is not the correct path, then what is an appropriate approach? Andrew Rutherford Department of Psychology Keele University email - a.rutherford@ psy.keele.ac.uk