In my recent reponse to Arturo Marti-Carvajal regarding how to know if a
result is a true negative or a false negative, there is an incorrect
statement in the second sentence. I said, "Statistically nonsignificant
findings with 80% power will be true negatives 80% of the time and false
negatives 20% of the time." I inadvertently confused sensitivity
(power) with negative predictive value. The sentence should say that
for a real difference (between treatments), with 80% power the
statistical test will give a true positive result 80% of the time and a
false negative result (not statistically significant) 20% of the time.
At the same time, with an alpha level of .05, no real difference will
give a true negative result (no statistically significant difference)
95% of the time and a false positive result 5% of the time. The
proportion of negative results that are true negatives or false
negatives is given by the negative predictive value. This is a more
complicated relationship that involves Bayes theorem and an estimate of
the pretest probability that the trial involves a real effect. This is
because, with less than 100% power, the statistical test is imperfect.
This is exactly analagous to an imperfect diagnostic test for which the
post test probability of disease is partly determined by the prevalence
of disease in the test population, according to Bayes' theorem. A few
months ago I posted a lengthy discussion of the parallel relationship
between satistical hypothesis testing and Bayesian analysis of imperfect
medical diagnostic tests. If anyone is interested in that, I can resend
the discussion to them. The core of the problem is that what Arturo
wants to know is: given a negative finding of no statistically
significant difference, what is the probability that this result is a
true negative rather than a false negative? He is asking for the
negative predictive value. A standard significance test cannot answer
that. Only a Bayesian calculation can give such an estimate. This is
the limitation of standard tests for statistical significance.
Thanks to Bernard North for catching my mistake. I believe the rest of
my response to Arturo is okay.
David L. Doggett, Ph.D., Medical Research Analyst
Health Technology Assessment and Information Service
ECRI, a non-profit health services research organization
5200 Butler Pike, Plymouth Meeting, PA 19462 USA
(610) 825-6000 ext 5509, FAX (610) 834-1275
[log in to unmask]
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
|