Hi Jon –
I think this is the nub of the whole issue, and basically relates to the concept of quality. Quality is often seen as being “as good as it can get”, but it
can also be seen as “good enough for purpose”. Spending weeks /months / years and loads of cash to find and obtain every last paper meets the first definition, but not necessarily the second. As has already been said in this discussion, it depends on whether
or not you are aiming for absolute scientific accuracy (and have the time / cash to do it).
I have always taken the pragmatic view that as guideline developers we do not have the time or resources to go for total recall, so we have opted for a “good
enough” approach that is spelled out for people looking at our stuff so they can judge for themselves whether or not they think it is adequate. That means using different standards for different questions. Searching for RCTs, for example, involves using a
tried and tested search filter. When looking for observational studies, a much broader and more sensitive search is needed. In all cases, though, it does mean looking at more than just Medline.
Cheers,
Robin
Scottish Intercollegiate Guidelines Network
Delta House | 50 West Nile Street | Glasgow G1 2NP
t:
0141 227 3298
e:
[log in to unmask]">[log in to unmask]
www.sign.ac.uk
The Scottish Intercollegiate Guidelines Network is part of Healthcare Improvement Scotland.
<image001.jpg>
I have little doubt that using databases other than medline increases the number of trials found. But, my interest, is what difference does this make?
The paper that Julie highlights appears to answer it, at least partially (but I'll reserve judgement until I've read it).
I suppose it what it your interest? Is it finding every article (irrespective of cost) or is it arriving at an accurate (another can of works) effect size?
On Tue, Oct 23, 2012 at 7:05 PM, Ahmed Abou-Setta, M.D. <[log in to unmask]> wrote:
Hi Jon,
The majority (if not all) studies answering this question relay that searching Medline
alone is not adequate. Here is one example: Suarez-Almazor ME, Belseck E, Homik J, Dorgan M, Ramos-Remus C. Identifying clinical trials in the medical literature with electronic databases: MEDLINE alone is not enough. Control Clin Trials. 2000 Oct;21(5):476-87
(http://www.embase.com/info/UserFiles/Files/medline_is_not_enough_article.pdf).
Also guidance from Carol Lefebvre (one of the members on this listserv) and the Cochrane
Handbook:
“A search of MEDLINE alone is not considered adequate. A systematic review showed that
only 30% - 80% of all known published randomized trials were identifiable using MEDLINE (depending on the area or specific question) (Dickersin 1994). Even if relevant records are in MEDLINE, it can be difficult to retrieve them (Golder 2006, Whiting 2008).
Going beyond MEDLINE is important not only for ensuring that as many relevant studies as possible are identified but also to minimize selection bias for those that are found. Relying exclusively on a MEDLINE search may retrieve a set of reports unrepresentative
of all reports that would have been identified through a comprehensive search of several sources.”
Hope this helps.
Ahmed
From: Evidence
based health (EBH) [mailto:[log in to unmask]]
On Behalf Of Jon Brassey
Sent: Tuesday, October 23, 2012 12:54 PM
To:
[log in to unmask]
Subject: Re: Medline based systematic review
I'm more interested in the 2) question. I appreciate that any studies found in systematic reviews are not in medline (and/or PubMed) but - in reality, does it
make a huge amount of difference? The reference found by Julie (http://www.hta.ac.uk/project/1099.asp) suggests (and I've only based this on the summary) that it doesn't make a big difference.
They state:
Systematic reviews that are based on a search of English language literature that is accessible in the major bibliographic databases will often produce results
that are close to those obtained from reviews based on more comprehensive searches that are free of language restrictions.
I imagine the article will unpick the medline/pubmed versus other 'major bibliographic databases' but I may be disappointed.
On Tue, Oct 23, 2012 at 6:39 PM, Ahmed Abou-Setta, M.D. <[log in to unmask]> wrote:
Hi Jon,
For clarification, do you mean ‘studies included in PubMed but that are not Medline-indexed’
or do you mean ‘studies found in other databases (e.g. Embase, Central, Cinahl, etc.) but that are not Medline-indexed’?
Both questions are valid and in my mind could be:
1)
What is the difference between Medline and Pubmed?
2)
Why should be search outside Medline?
Ahmed
From: Evidence
based health (EBH) [mailto:[log in to unmask]]
On Behalf Of Jon Brassey
Sent: Tuesday, October 23, 2012 12:13 PM
To:
[log in to unmask]
Subject: Medline based systematic review
I apologise if this turns out to be a stupid question. But can people point me to any systematic review based meta-analysis that has been repeated with all non-Medline
articles removed? In other words you're comparing a meta-analysis based on 'all' RCTs versus those just in Medline.
--
--