We in technology assessment look forward with great interest to the results
of Benjamin Djulbegovic's compilation of RCTs that changed medicine. These
good examples are useful in promoting EBM. At the same time, it would be
interesting to compile a list of medical interventions that were so
obviously and spectacularly successful that RCTs were never seriously
considered. The most commonly used example of this is the discovery of
penicillin. I suspect another is the use of X-rays to diagnose and plan
treatment for bone fractures (although there may well have been some RCTs
for this, I don't know). I think one of the reasons there is so much poorly
controlled medical research is that medical researchers tend to
optimistically hope that their pet innovation will turn out to be so
spectacularly successful that the tough work of RCTs will be unnecessary, or
can be left to others after they have claimed priority for the innovation.
It would be instructional for them to see the type of rare situations they
aspire to.
Also, examination of these RCT-unnecessary discoveries (let's call them
un-RCT situations) will be useful for recognizing situations when an RCT
might not be necessary for a decision. It will never be practical to carry
out large double-blind RCTs for every new development and minor innovation
on a previously proven intervention. Yet, in this era of cost-conscious
payers (private as well as public sector), there is a knee-jerk tendency to
refuse payment for any intervention that can't show conclusive RCT "proof".
This introduces the oft ignored potential for Type II error. I understand
that up until now people have worked so hard to get the need for RCTs
recognized that many people are wary of even discussing situations where
RCTs are unnecessary, but the fact remains that the real task is to
understand when an RCT is definitely required, and when other evidence is
trustworthy enough that an RCT would be unnecessary or even unethical.
Balancing the RCT success examples with the un-RCT success examples will be
most educational for us all. It will address the two main problems we have
in technology assessment: 1) not enough RCTs (give us the ammunition to
fight for more), and 2) how do we recognize and handle analysis and
decisions in un-RCT situations.
David L. Doggett, Ph.D.
Senior Medical Research Analyst
Technology Assessment Group
ECRI, a non-profit health services research organization
5200 Butler Pike
Plymouth Meeting, PA 19462-1298, USA
Phone: +1 (610) 825-6000 ext.5509
Fax: +1(610) 834-1275
E-mail: [log in to unmask]
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
|