With fewer than an infinite number of patients, one can never be certain
of a true negative. Statistically nonsignificant findings with 80%
power will be true negatives 80% of the time and false negatives 20% of
the time. If you want fewer false negatives, then you have to add more
patients (or decrease the variance, or lower the alpha level). As the
number of patients approaches infinity (or variance approaches 0, or
alpha approaches 1.0), the power approaches 100% asymptotically. Like
imperfect diagnostic tests, statistical significance tests (including
confidence intervals) are never 100% correct. It's all probability.
I believe what you really are asking is: how do we tell a truly negative
result from a merely inconclusive one? A truly negative result occurs
if there is reasonable power (say 80% or better) to detect the smallest
meaningful clinical effect, and yet the result is a negative effect or a
small but statistically nonsignificant positive effect. An inconclusive
result occurs if there is a positive but not statistically significant
result, but there was not sufficient power to detect the smallest
meaningful clinical effect with statistical significance even if it
occurred (power less than 80%). Clearly a power analysis is necessary
to know the meaning of a finding of statistical nonsignificance (even if
judged by confidence intervals).
The criteria for a useful alpha level, useful power, and meaningful
clinical effect will be unique to each situation depending on the
consequences of false positive and false negative results and the costs
and usefulness of the clinical effect. The traditional rule is alpha of
0.05 (same as 95% confidence interval) and beta of 0.20 (power of 80%),
but this is not universally appropriate. The smallest meaningful
clinical effect size is highly situational. Inexpensive, safe therapies
that affect large numbers of people (for example aspirin) may have very
small effect sizes that are useful (1% or less). Expensive or risky
therapies may need very large effect sizes (50% or larger) to be
considered worthwhile.
David L. Doggett, Ph.D., Medical Research Analyst
Health Technology Assessment and Information Service
ECRI, a non-profit health services research organization
5200 Butler Pike, Plymouth Meeting, PA 19462 USA
(610) 825-6000 ext 5509, FAX (610) 834-1275
[log in to unmask]
> -----Original Message-----
> From: Arturo Marti-Carvajal [SMTP:[log in to unmask]]
> Sent: Tuesday, January 19, 1999 12:51 PM
> To: [log in to unmask]
> Subject: Negative Results
>
> Dear members:
>
> Continuing with this Socratic dialogue, do I ask, which would they be,
> then, the requirements to consider to an investigation like true
> negative?". It would be very useful if of here a checkup list emerged
> to
> consider to a study as true negative.
>
> Arturo
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
|