WARNING: This is a stream-of-consciousness discussion, based on some
uncertainty about the concept of uncertainty of the uncertainty.
As a researcher, I could see two degrees of uncertainty, e.g. if we were to
do a study of studies and characterize the uncertainty reported in other
studies.
As a clinician, I have a hard time distinguishing the 2 layers of
uncertainty you allude to. When the information is uncertain, it is
uncertain. Am I certain or uncertain that the information is uncertain? If
there is so much evidence that the information is certain, then the only
uncertainty is about the degree of effect rather than effect vs. no effect.
So perhaps there are two degrees of uncertainty regarding the conclusions
about a dichotomous outcome:
certainty or uncertainty regarding whether a difference occurs
certainty or uncertainty regarding the amount of that difference
So if there is an NNT of 6 with a narrowish confidence interval (95% CI for
NNT 4 to 10), assuming the underlying evidence is valid, then I am certain
the treatment is effective and uncertain whether the degree of
effiectiveness is in the category of highly effective or moderately
effective.
As a physician making treatment decisions, I balance many issues including
efficacy, safety, patient preference, etc.
To simplify medical decision-making I might dichotomize uncertainty so that
I am actually considering "known" benefits, "potential" benefits, known
harms and potential harms.
I don't use an arbitrary number to distinguish known from potential, but
rather a gestalt of the available evidence.
The NNT (point estimate) and confidence intervals are part of the balance
but it generally is a gestalt and not a totally number-driven decision.
If giving advice to drink more to a patient with diarrhea has an NNT of 100
or even 1,000 to prevent an emergency department visit for dehydration with
narrow or wide confidence intervals (or in the absence of evidence is highly
uncertain whether it makes a difference at all), I would still give that
advice based on might help/no apparent harm/standard practice/seems
reasonable. I could make a theoretical argument about potential harms
(patient misunderstanding leading to excessive or improper drinking) but I
would accept this practice of "common sense" advice until seeing evidence to
the contrary.
If considering a toxic drug for a therapeutic regimen, I would want a lot
more evidence and certainty before making such a recommendation.
In statistical interpretation, p = 0.06 does not read to me as "no effect"
and p = 0.049 does not read to me as "proof of a difference" -- it helps me
have a sense of the uncertainty. Likewise, as much of clinical medicine is
a gray zone, I look more for degrees of uncertainty than an absolute cutoff.
|