 |
 |
Thanks, Pablo, that is illuminating. It is also reassuring for those who do rapid reviews. The conclusion seems to be that at least most of the time, a less than comprehensive search will give us the same outcome as a comprehensive search. Unfortunately, there may be occasions that this is not so. The trouble is, we cannot tell when those occasions will occur. Dr Kev (Kevork) Hopayian, MD FRCGP General Practitioner, Leiston, Suffolk Hon Sen Lecturer, Norwich Medical School, University of East Anglia Primary Care Tutor, Suffolk RCGP Clinical Skills Assessment examiner http://www.angliangp.org.uk/ On 25 Oct 2012, at 07:48, Pablo Alonso Coello <[log in to unmask]> wrote: > This study might be of interest regarding the performance of just MEDLINE for keeping up to date. > > > We compared the performance of McMaster Premium LiteratUre Service (PLUS) and Clinical Queries (CQs) to that of the Cochrane Controlled Trials Register, MEDLINE, and EMBASE for locating studies added during an update of reviews. From a sample of 98 updated Cochrane reviews the recall rates for all new trials by database was very high (even applying Clinical Queries filters). New studies in updated reviews were the reference standard. Overlapping was explored. > > PLUS included less than a quarter of the new studies in Cochrane updates, but most reviews appeared unaffected by the omission of these studies. If this is the case then using MEDLINE seems to work rather well for updating. > > > Have a look below. > > http://www.ncbi.nlm.nih.gov/pubmed/21856121 > > Overall relative recall rates for all new trials, by database, > were highest for CCTR at 95%, followed by > MEDLINE (90%), EMBASE (86%), and PLUS (23%). > The overall relative recall rates were substantially unaffected > by the exclusion of eight unpublished newly > added trials. Overlap existed between databases with only > 4% (n 524) of trials indexed exclusively in a single > database (MEDLINE, n5 1; EMBASE, n5 3; and > CCTR, n 520). > > Relative recall rates were not materially affected by the > application of CQ-sensitive filters for therapy (randomized > trials). CQ-filtered relative recall rates for MEDLINE were > 89% (vs. 90% unfiltered) and 84% for CQ-filtered EMBASE > (vs. 86% unfiltered). > > > Pablo Alonso Coello > Iberoamerican Cochrane Centre > Institute of Biomedical Research (IIB Sant Pau) Barcelona > C/ Sant Antoni Maria Claret 171 > 08041 Barcelona | España > [log in to unmask] | Skype: tartanedo > T: (+34) 93 553 78 14 | F: (+34) 93 553 78 09 > > De: Evidence based health (EBH) [[log in to unmask]] en nom de Ahmed Abou-Setta, M.D. [[log in to unmask]] > Enviat el: dimecres, 24 / octubre / 2012 18:20 > Per a: [log in to unmask] > Tema: Re: Medline based systematic review > > Hi Hilda, > > First of all let me apologize since my table become unreadable. Here is it as a picture. > <image001.png> > > Now, to clarify, we know how many citations the reviewers included can be found in PubMed. What we don’t know is if the same information is duplicated across all the publications. In other words, if you grabbed any review I can tell you which citations are available in PubMed versus only found in other DBs. What I can’t tell you without reading all the citations is how often for example the primary outcome was reported in both the PubMed citation vs. only in the non-PubMed citation vs. present in both the PubMed citation and the non-PubMed Citations. This comes back to the policy of study results being split across several publications (citations). Usually the primary outcome of the study (e.g. mortality rates) will be reported in all the publications, but I have worked with some studies that have over 20 companion studies and some of the most vital information to my review was reported only in small secondary publications. > > Ahmed > > > -----Original Message----- > From: Evidence based health (EBH) [mailto:[log in to unmask]] On Behalf Of Bastian, Hilda (NIH/NLM/NCBI) [C] > Sent: Wednesday, October 24, 2012 11:09 AM > To: [log in to unmask] > Subject: Re: Medline based systematic review > > G'day Ahmed! > > Thanks – so we don't know the proportion of studies in PubMed from this, if I'm understanding that right. > > Hilda > > From: "<Ahmed Abou-Setta>", "M.D." <[log in to unmask]<mailto:[log in to unmask]>> > Reply-To: "Ahmed Abou-Setta, M.D." <[log in to unmask]<mailto:[log in to unmask]>> > Date: Wednesday, October 24, 2012 12:00 PM > To: "[log in to unmask]<mailto:[log in to unmask]>" <[log in to unmask]<mailto:[log in to unmask]>> > Subject: Re: Medline based systematic review > > > Hi Hilda, > > > > I looked at included citations as opposed to included studies because since some studies had more than one citation. There were 385 studies and 1073 citations included in the 9 reviews, which averages 1.3 citations per study. In case anyone is interested in seeing a breakdown: > > > Reviews > > Studies > > Total number of Citations > > Citations found in PubMed > > % > > Review #1 > > 3 > > 6 > > 3 > > 50% > > Review #2 > > 40 > > 64 > > 28 > > 44% > > Review #3 > > 4 > > 4 > > 3 > > 75% > > Review #4 > > 16 > > 74 > > 63 > > 85% > > Review #5 > > 90 > > 448 > > 212 > > 47% > > Review #6 > > 24 > > 28 > > 14 > > 50% > > Review #7 > > 41 > > 78 > > 44 > > 56% > > Review #8 > > 8 > > 13 > > 9 > > 69% > > Review #9 > > 159 > > 358 > > 143 > > 40% > > Total > > 385 > > 1073 > > 519 > > 48% > > > > > This is all to stimulate discussion. For accurate results, we would have to check all reviews in the Cochrane Library, and when multiple citations are available for the same study, we would have to check the outcome data from the studies in Medline vs. only in other DBs to see if any additional information was provided in publications indexed outside of Medline. > > > > Ahmed > > > > -----Original Message----- > From: Evidence based health (EBH) [mailto:[log in to unmask]] On Behalf Of Bastian, Hilda (NIH/NLM/NCBI) [C] > Sent: Tuesday, October 23, 2012 5:59 PM > To: [log in to unmask]<mailto:[log in to unmask]> > Subject: Re: Medline based systematic review > > > > Hi - did you look for unique included studies or citations? > > > > Hilda > > ________________________________________ > > From: Ahmed Abou-Setta, M.D. [[log in to unmask]<mailto:[log in to unmask]>] > > Sent: Tuesday, October 23, 2012 4:16 PM > > To: [log in to unmask]<mailto:[log in to unmask]> > > Subject: Re: Medline based systematic review > > > > Hi Jon, > > > > Fair enough… so I just did a simple experiment. On the Cochrane (Wiley) website they have a section called “Highlighted New and Updated Cochrane Reviews”. So I wanted to see how many of these included citations were available in PubMed. The results is that out of 1073 included citations, only 519 (48%) were available in PubMed. You would still want to remove the ones that are indexed in PubMed and not Medline and so the final tally will be even less. They ranged from 40% to 85% for each review. So the final answer would probably be that looking only at Medline will only give you a biased estimate of the truth. > > > > Ahmed > > > > > > From: Evidence based health (EBH) [mailto:[log in to unmask]]<mailto:[mailto:[log in to unmask]]> On Behalf Of Jon Brassey > > Sent: Tuesday, October 23, 2012 1:54 PM > > To: [log in to unmask]<mailto:[log in to unmask]> > > Subject: Re: Medline based systematic review > > > > Hi Ahmed, > > > > I have little doubt that using databases other than medline increases the number of trials found. But, my interest, is what difference does this make? > > The paper that Julie highlights appears to answer it, at least partially (but I'll reserve judgement until I've read it). > > > > I suppose it what it your interest? Is it finding every article (irrespective of cost) or is it arriving at an accurate (another can of works) effect size? > > > > BW > > > > jon > > On Tue, Oct 23, 2012 at 7:05 PM, Ahmed Abou-Setta, M.D. <[log in to unmask]<mailto:[log in to unmask]<mailto:[log in to unmask]:[log in to unmask]>>> wrote: > > Hi Jon, > > > > The majority (if not all) studies answering this question relay that searching Medline alone is not adequate. Here is one example: Suarez-Almazor ME, Belseck E, Homik J, Dorgan M, Ramos-Remus C. Identifying clinical trials in the medical literature with electronic databases: MEDLINE alone is not enough. Control Clin Trials. 2000 Oct;21(5):476-87 (http://www.embase.com/info/UserFiles/Files/medline_is_not_enough_article.pdf). > > > > Also guidance from Carol Lefebvre (one of the members on this listserv) and the Cochrane Handbook: > > > > “A search of MEDLINE alone is not considered adequate. A systematic review showed that only 30% - 80% of all known published randomized trials were identifiable using MEDLINE (depending on the area or specific question) (Dickersin 1994). Even if relevant records are in MEDLINE, it can be difficult to retrieve them (Golder 2006, Whiting 2008). Going beyond MEDLINE is important not only for ensuring that as many relevant studies as possible are identified but also to minimize selection bias for those that are found. Relying exclusively on a MEDLINE search may retrieve a set of reports unrepresentative of all reports that would have been identified through a comprehensive search of several sources.” > > > > Hope this helps. > > > > Ahmed > > > > > > From: Evidence based health (EBH) [mailto:[log in to unmask]<mailto:[log in to unmask]>]<mailto:[mailto:[log in to unmask]:[log in to unmask]]> On Behalf Of Jon Brassey > > Sent: Tuesday, October 23, 2012 12:54 PM > > To: [log in to unmask]<mailto:[log in to unmask]<mailto:[log in to unmask]:[log in to unmask]>> > > Subject: Re: Medline based systematic review > > > > Hi Ahmed, > > > > I'm more interested in the 2) question. I appreciate that any studies found in systematic reviews are not in medline (and/or PubMed) but - in reality, does it make a huge amount of difference? The reference found by Julie (http://www.hta.ac.uk/project/1099.asp) suggests (and I've only based this on the summary) that it doesn't make a big difference. They state: > > > > Systematic reviews that are based on a search of English language literature that is accessible in the major bibliographic databases will often produce results that are close to those obtained from reviews based on more comprehensive searches that are free of language restrictions. > > > > I imagine the article will unpick the medline/pubmed versus other 'major bibliographic databases' but I may be disappointed. > > > > BW > > > > jon > > > > > > > > On Tue, Oct 23, 2012 at 6:39 PM, Ahmed Abou-Setta, M.D. <[log in to unmask]<mailto:[log in to unmask]<mailto:[log in to unmask]:[log in to unmask]>>> wrote: > > Hi Jon, > > > > For clarification, do you mean ‘studies included in PubMed but that are not Medline-indexed’ or do you mean ‘studies found in other databases (e.g. Embase, Central, Cinahl, etc.) but that are not Medline-indexed’? > > > > Both questions are valid and in my mind could be: > > > > > > 1) What is the difference between Medline and Pubmed? > > > > 2) Why should be search outside Medline? > > > > Ahmed > > > > > > From: Evidence based health (EBH) [mailto:[log in to unmask]<mailto:[log in to unmask]>]<mailto:[mailto:[log in to unmask]:[log in to unmask]]> On Behalf Of Jon Brassey > > Sent: Tuesday, October 23, 2012 12:13 PM > > To: [log in to unmask]<mailto:[log in to unmask]<mailto:[log in to unmask]:[log in to unmask]>> > > Subject: Medline based systematic review > > > > Hi All, > > > > I apologise if this turns out to be a stupid question. But can people point me to any systematic review based meta-analysis that has been repeated with all non-Medline articles removed? In other words you're comparing a meta-analysis based on 'all' RCTs versus those just in Medline. > > > > Best wishes > > > > jon > > -- > > Jon Brassey > > TRIP Database > > http://www.tripdatabase.com > > Find evidence fast > > > > > > > > > > > > -- > > Jon Brassey > > TRIP Database > > http://www.tripdatabase.com > > Find evidence fast > > > > > > > > > > > > -- > > Jon Brassey > > TRIP Database > > http://www.tripdatabase.com > > Find evidence fast