Dear Allstat members
I would be grateful if any of you could please advise me on how to
perform a retrospective power calculation for Fisher's Omnibus test.
This test takes n one-sided p-values and combines them to provide one
overall p-value. It uses the fact that -2 times the natural logarithm of
a uniformly distributed random variable has a chi-squared distribution
with two degrees of freedom - therefore under H0 the sum of the n
independent log-transformed one-sided p-values has a chi-squared
distribution with 2n degrees of freedom (Haccou, P. & Meelis, E. (1992).
Statistical Analysis of Behavioural Data. An Approach Based on
Time-Structured Models. - Oxford University Press). I am using the test
to compare the results of a series of behavioural experiments (where the
number of fish within a fixed area is varied 20 times and the size and
number of shoals recorded at each fish 'density') to the results of two
models of shoaling behaviour in which the fish densities are replicated
many times. One model is expected to match the experimental results
pretty well, the other is expected to be a poor fit because the 'fish'
are merely performing a random 'walk' within the arena and have no
programmed shoaling tendency (a null model). We used a Monte Carlo
technique to obtain a p-value for each comparison (experiment vs
shoaling model, experiment vs null model) at each density, then used
Fisher's Omnibus test to combine the 20 p-values obtained (one at each
density), for each comparison. As expected, the combined p-values for
the comparisons between the experimental results and the null model were
significant (p<0.001 for both shoal size and shoal number), whereas the
combined tests comparing the experimental results to the shoaling model
were not significant (P = 0.825, P = 0.430 for shoal size and number
respectively). One reviewer has pointed out that we should provide a
power calculation for the non-significant results. My problem is that
while I have software (NQuery Advisor 5.0, Sample Power 2.0) that will
perform power calculations for chi-squared tests, the programs generally
require proportions for two groups to be entered. All I have is an
f-statistic and degrees of freedom (40) for each test. From tables I can
see that the critical f for testing at the 5% level with 40 df is
55.758. Can I in some way compare my derived f-statistics to this figure
in order to compute power? Any advice you have would be greatly
appreciated.
Incidentally, this analysis comes from my behavioural ecology PhD thesis
- the academic unit is not branching out into fish behaviour!
Thanks in advance
Liz Hensor
Dr Elizabeth M A Hensor PhD
Data Analyst
Academic Unit of Musculoskeletal and Rehabilitation Medicine
36 Clarendon Road
Leeds
West Yorkshire
LS2 9NZ
Tel: +44 (0) 113 3434944
Fax: +44 (0) 113 2430366
[log in to unmask]
|