Hi. I'm new to this list. I sent these queries to a list member some time ago, so this may be familiar. However, I was not subscribed to the list at that time, so I'm not sure if this topic was discussed.


I'm involved in a debate in which someone is claiming that there is no meaningful objective evidence that the results of RCTs and 'outcomes research' and other observational studies differ significantly when evaluated in the context of a given intervention. This is to say that he believes that outcomes research is essentially equally accurate and meaningful to RCTs despite it's lower position in the 'Levels of Evidence' hierarchy. He is basing this opinion upon only one study, though it was published in the NEJM (comparing RCTs and observational studies):

http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=pubmed&dopt=Abstract&list_uids=10861324&query_hl=1

A comparison of observational studies and randomized, controlled trials.

Benson K, Hartz AJ.

Department of Family Medicine, University of Iowa College of Medicine, Iowa City 52242-1097, USA.

BACKGROUND: For many years it has been claimed that observational studies find stronger treatment effects than randomized, controlled trials. We compared the results of observational studies with those of randomized, controlled trials. METHODS: We searched the Abridged Index Medicus and Cochrane data bases to identify observational studies reported between 1985 and 1998 that compared two or more treatments or interventions for the same condition. We then searched the Medline and Cochrane data bases to identify all the randomized, controlled trials and observational studies comparing the same treatments for these conditions. For each treatment, the magnitudes of the effects in the various observational studies were combined by the Mantel-Haenszel or weighted analysis-of-variance procedure and then compared with the combined magnitude of the effects in the randomized, controlled trials that evaluated the same treatment. RESULTS: There were 136 reports about 19 diverse treatments, such as calcium-channel-blocker therapy for coronary artery disease, appendectomy, and interventions for subfertility. In most cases, the estimates of the treatment effects from observational studies and randomized, controlled trials were similar. In only 2 of the 19 analyses of treatment effects did the combined magnitude of the effect in observational studies lie outside the 95 percent confidence interval for the combined magnitude in the randomized, controlled trials. CONCLUSIONS: We found little evidence that estimates of treatment effects in observational studies reported after 1984 are either consistently larger than or qualitatively different from those obtained in randomized, controlled trials.





I am well aware of the theoretical reasons for which RCTs are considered more reliable than outcomes research and occupy a higher level in the hierarchy. What I'm looking for is something more objective, i.e., are there any important studies which demonstrate the value of RCTs over any and/or all other study designs in an objective, practical sense rather than in theoretical construct?

To be more specific, here are a couple of statements which I think may be
refutable:

"There is no evidence that I am aware of that demonstrates a meaningful
difference between RCT and outcomes studies for the same of similar
condition."


"In the modern era (within the last 15 years, from what I've read, there is
NO difference in conclusions drawn between outcomes studies and RCT. (when
studying similar conditions)."

"From what I've read...the only justification for the claim that RCT are in
a meaningful way superior than outcomes studies extrapolated from studies
done in the 1940's early 50s."

"from what I've read there is no convincing evidence that RCTs produce
superior (in real terms) evidence than case series>"







Are the above statements correct?



Thanks,

 Russ

_________________________________________________________________
On the road to retirement? Check out MSN Life Events for advice on how to get there! http://lifeevents.msn.com/category.aspx?cid=Retirement