Here is a formal reason that can justify power analysis, whether ex-ante
or ex-post, even when the p value is below 5% - it is of course not
pragmatic if the p value is very small:
Even when the p value is below 5% then power comparisons can be needed
in the following case. Power is the main formal basis for selecting a
test. When there is no uniformly most powerful test then one has to fix
the type of alternative one wishes to look at and the best test is the
one that minimizes the type II error. For instance, for a noninferiority
test the default alternative hypothesis is equivalence of the two
treatments.
Now assume that one suspects that the statistician did not choose the
test that yields the lowest type II error, i.e. that another test yields
a lower type II error. It is namely not true, that a test that yields a
lower type II error will automatically give a lower p value. In other
words it can happen that the statistician gets a p value below 5% while
the test that gives the lowest (or a lower) type II error gives a p
value above 5%. The way out of this dilemma is to either report the
results of the different standard tests or to perform a power
calculation to justify the use of the specific test via arguing in terms
of its low type II error.
karl
--
---------------------------------------------------------------------
Karl Schlag
Professor Tel: +34 93 542 1493
Department of Economics and Business Fax: +34 93 542 1746
Universitat Pompeu Fabra email: [log in to unmask]
Ramon Trias Fargas 25-27 www.iue.it/Personal/Schlag/
Barcelona 08005, Spain room: 20-221 Jaume I
|