Dear James,
I agree that non-randomised designs could be more usefully incorporated into
Cochrane reviews (though I'm not sure about the virtue of combining
non-randomised studies in meta-analyses). I think there's a growing
interest in how to do this effectively and systematically.
In some cases, non-randomised studies are included in Cochrane reviews. See,
for example, the review about smoke detectors or our review about personal
assistance (paid carers) for the elderly. In both cases, it was unrealistic
to expect there would be lots of RCTs and the reviewers included studies
with contemporaneous comparison groups.
But for reviews that are limited to RCTs, I don't know of good guidance on
handling excluded studies in a way that conveys more information than the
reason for exclusion. I suspect editors may be hesitant to allow reviewers
to say much about these studies when reviewers haven't developed protocols
for synthesising non-randomised data, there's no clear reason for including
some studies but not others in the 'Excluded Studies' section, search
strategies were designed for RCTs, etc.
I wonder if anybody has come across good suggestions or guidelines for
managing excluded studies, particularly when there aren't many included
studies to guide practice?
Best wishes,
Evan
Evan Mayo-Wilson
Departmental Lecturer
Centre for Evidence-Based Intervention
Barnett House
32 Wellington Square
Oxford OX1 2ER, UK
+44 (0) 1865 280339 (phone)
+44 (0) 1865 270324 (fax)
[log in to unmask]
-----Original Message-----
From: Evidence based health (EBH)
[mailto:[log in to unmask]] On Behalf Of James Osborne
Sent: 29 April 2009 17:17
To: [log in to unmask]
Subject: Re: Systematic reviews with only one study - or single-study
comparisons
When demonstrating the Cochrane Library and Cochrane reviews to students and
NHS staff, it is amazing how many times the review they randomly pick on
ends up excluding all or virtually all RCTs. When this happens they really
don't see the point of such a 'blank' message to users. Nothing quite like
an anectdote - is there a statistic on the percentage of Cochrane
meta-analyses
that do this?
I feel Cochrane take an overly purist approach when there is either 0 or 1
RCTs included (on quality grounds) in a meta-analyis for any particular
outcome. In such cases where there is such a dearth of evidence, it could
present the lesser quality RCT or non-RCT results as a sub-group analysis -
obviously with caveats.
I'm sure the purists amongst us will abhor the concept of attempting let
alone publishing such a mixed meta-analysis. The only online resource that
I'm aware of that tries this difficult trick of graphically summarising ALL
the available evidence is the 'Evidence Matters' resource from Canada. If
anyone
on the list has access to their resource (we don't) it would be interesting
to know how helpful such an approach really is.
James Osborne
Clinical Effectiveness Coordinator
University Hospitals Bristol NHS Foundation Trust
[PS]
I have no relationship with or conflict of interest in Evidence Matters (or
for that matter Cochane), I've just seen it at a conference and tried their
demo
version and instinctively liked their approach.
|