Dear Steve,
I'm sympathetic with the the idea that most articles we ask students to
read should not be fatally flawed. We usually try to give students
reasonably good studies for the first few particularly. There are
usually enough flaws even in "good" trials to get students thinking. But
a next step is to compare articles and ask which students would prefer
to base their decision on. For example, we've used abstracts of cohort
studies versus RCTs of folate for cardiovascular disease. But a good
further step would be to compare two RCTs of the same question.
Cheers
Paul Glasziou
Steve Simon wrote:
> Terry Shaneyfelt wrote:
>> Does any one have a good example of a fairly flawed therapy article
>> for a course that I teach in EBM. Seems most of the articles I find
>> arent too badly designed. Students always want to see some bad
>> articles to critique.
>
> The cynic in me says that you need to spend more time reading journal
> articles.
>
> But I'm not much of cynic actually, and if anything I would discourage
> this activity.
>
> I've always disliked the concept of finding an article that you can
> trash. I think it creates a sense of cynicism in students and also
> develops the false impression that critical appraisal means literally
> being critical of everything you see. It also encourage black and
> white thinking while most studies are on a continuum of the
> persuasiveness of the evidence.
>
> I've heard the comment made more than once along the lines of read the
> methods section first and if you see something wrong then stop reading
> the article. I would disagree with this advice. I would say that you
> still read the methods section first, but only to assess the degree of
> persuasiveness that this article is likely to have. Then look at the
> results. Finally, see if there is any corroborating evidence
> (plausible mechanism, replication of an early result, strong effect,
> dose-response pattern, specificity of effect, and so forth). Conflicts
> of interest factor in here also. If the evidence is highly objective,
> then it doesn't matter if there was pharmaceutical money involved. But
> a borderline finding becomes that much less persuasive when there is
> such a conflict.
>
> I do mention a few bad studies in my book (Statistical Evidence in
> Medical Trials). For example, there is the famous study by Linus
> Pauling about vitamin C that prospectively evaluated patients with
> terminal cancer on vitamin C but used for a control group patients
> selected from a death certificate database.
>
> Supplemental ascorbate in the supportive treatment of cancer:
> prolongation of survival times in terminal human cancer. Ewan Cameron,
> Linus Pauling. Proc Natl Acad Sci USA 1976: 73(10); 3685-89.
> www.pubmedcentral.nih.gov/picrender.fcgi?artid=431183&blobtype=pdf
>
> But those examples are to illustrate a specific lesson. I would be
> uncomfortable asking students to find the flaw in these studies.
>
> You should critique studies that are representative of what students
> will encounter in the real world. If those articles are mostly well
> designed, then so be it.
--
Paul Glasziou
Director, Centre for Evidence-Based Medicine,
Department of Primary Health Care,
University of Oxford www.cebm.net
ph - +44-1865-289298 fax +44-1865-289287
|