Ben Djulbegovic posted:
> What appears (at least to me in this phase of thinking of this problem),
>that acceptance of dramatic results from a small clinical series will have
>to be dealt on the case by case basis, taking into considerations all other
>available biologic and clinical information. But, if this is the case
>indeed, then this means that the GENERAL SOLUTION to the problem of
>hierarchy of evidence is not possible, which in turn questions fundamental
>principle of EBM (that is, that some findings are closer to the truth than
>others, and we can decipher it from underlying design of the study)?
Seems to me that there are two questions that both need to be addressed.
First, how much precision is required for a decision (which determines
sample size). Gehan's 1961 paper presents an interesting approach, starting
by using small samples to rule out Type II error and then addressing Type I.
If solutions are trivial in terms of cost, risks, complexity, etc., then one
might not need all that large an impact to justify imposing the intervention
(eg: cooking ground meet thoroughly to prevent enteroinvasive E. coli
infections). If interventions are costly, risky, hazardous or very
unpleasant, then larger impact is needed to justify imposing them. Thus,
smallest difference one wants ability to detect is a clinical or economic
rather than statistical consideration. That Gehan paper is:
PARADIGM Database Search, 27 July 99:
-------------------------------------
Gehan EA
The Determination of the Number of Patients Required in
a Preliminary & a Follow-Up Trial of a New Chemotherapeutic Agent
J CHRON DIS 1961;13(4):346-53
A sequential approach ("Multistage Trial") to evaluate
new agents proposed. Conditional probability of failure
in "n" subjects used to define 1'st sample size as
function of assigned value for Type-II error;if 0
success observed among "n" patients, then agent
considered not at least effective as assumed & trial
stops. If 1 or more successes is observed, then second
sample taken to bring total up to number required for a
specified Type-I error risk (from binomial approx. to
normal distribution calculation). Tables provided for
required 1'st & 2'nd sample at various levels of
effectiveness, Type-II & Type-I error. SEE ALSO Machin
& Campbell's book.
---COPYRIGHT, APPLIED EPIDEMIOLOGY---
Beyond this rests the second question: how valid is the study for its
intended purpose. This second question would be where study design and
validation criteria help to establish a strength-of-evidence hierarchy. And
that hierarchy could be different depending on whether one's objective is to
answer whether a treatment could work (efficacy, best addressed by suitably
blinded and randomized controlled trials) versus whether it does work
(effectiveness, perhaps better addressed by observational studies to see
what happends in everyday practice).
David.
David Birnbaum, PhD, MPH
Clinical Assistant Professor
Dept. of Health Care & Epidemiology
University of British Columbia, Canada
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
|