 |
 |
There is another issue here in the creation of very thorough literature reviews, be they called a meta analysis or a systematic review or whatever. Access to literature and the indexing of literature varies with the institutional affiliations of the individual who produces bibliographic content and the time on task one can spend in searching different databases to find content. I my public health guide I list the databases I suggest as useful for public health, and seperately for social work and yet another list of databases for finding statistical content. These are each very large lists and I even show in the social work section the number of hits in databases for a group of scoial work phrases. http://guides.temple.edu/c.php?g=78651&p=1090522 To search a group of the "best" of these databases and find citations in each to quality sources for the bibliography being constructed as well as weeding out duplication of the same citations from different databases is going to be a huge and very time consuming process, assuming there is a substantial body of literature on the topic of the bibliography. Hence the thoroughness of the bibliography may be limited due to the need of the producer to have the time to lead a normal life as well. I am finding this discussion to be most helpful and informative and I greatly thank all of you who have been contributing. I also would point out that often on topic content can often be found about a subject from databases that may seem completely unrelated to the subject of a research topic or bibliography. . Sincerely, David Dillard Temple University (215) 204 - 4584 [log in to unmask] Net-Gold http://groups.yahoo.com/group/net-gold http://listserv.temple.edu/archives/net-gold.html https://groups.io/org/groupsio/Net-Gold/archives http://net-gold.3172864.n2.nabble.com/ SPORT-MED https://www.jiscmail.ac.uk/lists/sport-med.html http://groups.yahoo.com/group/sports-med/ http://listserv.temple.edu/archives/sport-med.html On Wed, 19 Nov 2014, Dean Giustini wrote: > Hi Susan, Jon and others, > Recently, I have asked myself similar questions about expert/exhaustive searches. For example: What evidence is there that > exhaustive searches result in better research? more reliable research? What proof is there that this is true? > > For example, this paper was published recently: > * Li L, Tian J, Tian H, et al. Network meta-analyses could be improved by searching more sources and by involving a > librarian. J Clin Epidemiol. 2014 Sep;67(9):1001-1007. http://www.ncbi.nlm.nih.gov/pubmed/24841794 > > As an information professional, I was exciting to read that network meta-analyses can be improved by searching in more sources > and involving a librarian. I was hoping to read an evaluation of whether improved "better searches" changed the results of the > meta-analysis. But the authors could have looked at these studies, looked at the search strategies, improved them with "more > sources and a librarian" and then checked to see if the improvements changed the results. > > > That's the type of evidence I would like to see... > > Dean Giustini, MLS, MEd > UBC Biomedical Branch Librarian > Diamond Health Care Centre, VGH > 2775 Laurel Street, Vancouver BC > t: 604.875.4505 > [log in to unmask] > @giustini > > On 19 November 2014 08:53, Susan Fowler <[log in to unmask]> wrote: > For evidence I refer you to the guidelines offered by Cochrane, PRISMA, and IOM. They list the evidence that > supports their recommendations for thorough searching. > > On Wed, Nov 19, 2014 at 10:39 AM, Jon Brassey <[log in to unmask]> wrote: > Hi Susan, > > I would love to see some evidence around all this. You say 'should do a more exhaustive search' - what is that > based on? > > There have recently been a number of studies that demonstrate undertaking a meta-analysis using only a sample of > published trials (via modified search protocols) makes little difference (typically compared with a Cochrane > systematic review). > > Are you aware of any studies that have shown just doing a Medline search alters the outcome of a meta-analysis > based on a 'full' systematic review. I'm aware of studies that show that you miss trials by focusing on Medline - > but I'm not sure how often that leads to an altered outcome. It'd be a fascinating read. > > Thanks in advance. > > jon > > > > On Wed, Nov 19, 2014 at 4:24 PM, Susan Fowler <[log in to unmask]> wrote: > I didn't mean to sound so extreme. Of course we cannot find it all but we should do a more exhaustive > search then just medline and we should include grey lit resources. To me, if neither of those things > have been attempted - searching more then medline and searching grey lit, I would not define it as a > systematic review. > I believe definitions like,... > > A Systematic review “attempts to collate all empirical evidence that fits pre-specified > eligibility criteria in order to answer a specific research question. It uses explicit, > systematic methods that are selected with a view to minimizing bias, thus providing more > reliable findings from which conclusions can be drawn and decisions made.” Higgins JPT, Green > S (editors). Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 [updated > March 2011]. The Cochrane Collaboration, 2011. Available from www.cochrane-handbook.org., > Section 1.2.2 > > > specifies attempts for good reason. Researchers should to attempt to collate all empirical evidence knowing > they will not be able to but also knowing that they have to do more then search medline. > > > > > > Susan > > On Wed, Nov 19, 2014 at 10:07 AM, Jon Brassey <[log in to unmask]> wrote: > I am unaware of any systematic review that gets ALL the studies and I don't believe there is any > way of telling if you've got them all or not. As we now know (via AllTrials) is that > approximately 30-50% of trials are unpublished - many/most of these unpublished trials are > invisible to typical SR methods. > > If people search multiple databases and assume they find all trials that's simply wrong and > potentially dangerous. They may have found ALL published trials but that's entirely different from > ALL trials. > > I often work through the numbers to illustrate the issue. Assume there have been 20 trials and that > 40% (8 trials) have not been published. A search of Medline might find 10 of the 12 published trials > and an exhaustive search of multiple databases might find the other 2. > > So, a Medline systematic review is based on a sample of 10/20 trials (a 50% sample) while an > exhaustive search might be based on 12/20 (60% sample). I'd love to see some cost-benefit figures > around this. In other words, are the extra 2 trials worth the massive increase in database search > time. > > Best wishes > > jon > > > > On Wed, Nov 19, 2014 at 3:05 PM, Susan Fowler <[log in to unmask]> wrote: > If a "systematic review" reports only searching medline, it is NOT a systematic review > because the researchers have not collected ALL the studies that relate to their question. > -- > Susan Fowler, MLIS > Medical Librarian > Coordinator, Systematic Review Services > > Evidence at Becker: > http://beckerguides.wustl.edu/ebm > > Systematic Reviews Guide: > http://beckerguides.wustl.edu/SystematicReviews > > Becker Medical Library, Washington University in St. Louis > 314-362-8092 > [log in to unmask] > > On Wed, Nov 19, 2014 at 8:25 AM, Ansari, Mohammed <[log in to unmask]> wrote: > > Hi Tom, > > > > My understanding is only mine based on experience – I pen them down not looking at > the definitions you provide. Qualitatively each one of these is on a continuum > ---sometimes so poor that some may consider them not to be what they claim to be. So > actually it’s a matter of where one places their defining cut-off. Like sensitivity > and specificity there will be trade-offs. When we screen in SRs at abstract level we > consider the following criteria: > > > > At least reports: > > A research question/purpose > > Searching Medline > > Last search date > > And attempts to answer the question based on primary literature > > > > At full text level: > > Implicitly or explicitly indicate screening the search output against some > eligibility criteria > > Implicitly or explicitly appraise the validity and applicability of studies > (separating RCT from observational studies counts as some modicum of appraisal) > > Synthesizes results to reach a bottom line – not just descriptive paragraph by > paragraph summary of what X, Y and Z et al… > > Uses assessment of critical appraisal in synthesizing results to provide the best > available synthesis > > > > Meta-analysis: > > Statistical pooling of study summary data > > > > IPD: > > Statistical pooling of raw data from more than one study > > > > From: Evidence based health (EBH) [mailto:[log in to unmask]] On > Behalf Of Tom Jefferson > Sent: Wednesday, November 19, 2014 6:25 AM > To: [log in to unmask] > Subject: How do you define a systematic review, meta-analysis and IPD? > > > > Dear list members, I would be grateful if you could give me your views on the > definitions of what constitutes a systematic review, meta-analysis and IPD. I have > reported below definitions from the most popular sources of information (highest > Google algorithm position). I have numbered each definition 1 to 6 for ease of > commenting. > Thanks for your time. > > > Systematic review - Wikipedia: > > > 1. A systematic review (also systematic literature review or structured literature review, > SLR) is a literature review focused on a research question that tries to identify, > appraise, select and synthesize all high quality research evidence relevant to that > question > > and > > A systematic review aims to provide an exhaustive summary of current literature relevant > to a research question. The first step of a systematic review is a thorough search of the > literature for relevant papers. The Methodology section of the review will list the > databases and citation indexes searched, such as Web of Science, Embase, and PubMed, as > well as any hand searched individual journals. Next, the titles and the abstracts of the > identified articles are checked against pre-determined criteria for eligibility and > relevance. This list will always depend on the research problem. Each included study may > be assigned an objective assessment of methodological quality preferably using a method > conforming to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses > (PRISMA) statement (the current guideline)[5] or the high quality standards of Cochrane > collaboration.[6] > > (https://en.wikipedia.org/wiki/Systematic_review) > > Systematic review - Cochrane/PRISMA: > > 2. A systematic review is a review of a clearly formulated question that uses systematic > and explicit methods to identify, select, and critically appraise relevant research, and > to collect and analyze data from the studies that are included in the review. Statistical > methods (meta-analysis) may or may not be used to analyze and summarize the results of the > included studies. Meta-analysis refers to the use of statistical techniques in a > systematic review to integrate the results of included studies. > > (http://www.cochrane.org/glossary/5#letters) > > > > Meta-analysis - Wikipedia > > 3. In statistics, meta-analysis comprises statistical methods for contrasting and > combining results from different studies in the hope of identifying patterns among study > results, sources of disagreement among those results, or other interesting relationships > that may come to light in the context of multiple studies.[1] Meta-analysis can be thought > of as "conducting research about previous research." In its simplest form, meta-analysis > is done by identifying a common statistical measure that is shared between studies, such > as effect size or p-value, and calculating a weighted average of that common measure. This > weighting is usually related to the sample sizes of the individual studies, although it > can also include other factors, such as study quality. > > (https://en.wikipedia.org/wiki/Meta-analysis) > > Meta-analysis - Cochrane > > 4. The use of statistical techniques in a systematic review to integrate the results of > included studies. Sometimes misused as a synonym for systematic reviews, where the review > includes a meta-analysis. > > (http://www.cochrane.org/glossary/5#letterm) > > Individual patient data (IPD) - Cochrane > > 5. Individual patient data [In meta-analysis:] The availability of raw data for each study > participant in each included study, as opposed to aggregate data (summary data for the > comparison groups in each study). Reviews using individual patient data require > collaboration of the investigators who conducted the original studies, who must provide > the necessary data. > > (http://www.cochrane.org/glossary/5#letteri) > > > Individual patient data (IPD) - Bandolier > > 6. In systematic reviews this term refers to the availability of raw data for each study > participant in each included trial, as opposed to aggregate data (summary data for the > comparison groups in each study). Reviews using individual patient data require > collaboration of the investigators who conducted the original trials, who must provide the > necessary data. > > (http://www.medicine.ox.ac.uk/bandolier/booth/glossary/individual.html) > > -- > > Dr Tom Jefferson > Medico Chirurgo > GMC # 2527527 > > ______________________________________________________________________________________________________________________________ > > Confidentiality Statement - The contents of this e-mail, including its attachment, are > intended for the exclusive use of the recipient and may contain confidential or privileged > information. If you are not the intended recipient, you are strictly prohibited from > reading, using, disclosing, copying, or distributing this e-mail or any of its contents. > If you received this e-mail in error, please notify the sender by reply e-mail immediately > or the Privacy Office ([log in to unmask]) and permanently delete this e-mail > and its attachments, along with any copies thereof. Thank you. > > > > Avis de confidentialité – Ce courriel, y compris ses pièces jointes, s’adresse au > destinataire uniquement et pourrait contenir des renseignements confidentiels. Si vous > n’êtes pas le bon destinataire, il est strictement interdit de lire, d’utiliser, de > divulguer, de copier ou de diffuser ce courriel ou son contenu, en partie ou en entier. Si > vous avez reçu ce courriel par erreur, veuillez en informer immédiatement l’expéditeur ou > le bureau de la Protection des renseignements personnels > ([log in to unmask]), puis effacez le courriel ainsi que les pièces jointes > et toute autre copie. Merci. > > ______________________________________________________________________________________________________________________________ > > > > > > > > > > -- > Jon Brassey > Trip Database > http://www.tripdatabase.com > Find evidence fast > > > > > > > > > > > > -- > Jon Brassey > Trip Database > http://www.tripdatabase.com > Find evidence fast > > > > > > -- > Susan Fowler, MLIS > Medical Librarian > Coordinator, Systematic Review Services > > Evidence at Becker: > http://beckerguides.wustl.edu/ebm > > Systematic Reviews Guide: > http://beckerguides.wustl.edu/SystematicReviews > > Becker Medical Library, Washington University in St. Louis > 314-362-8092 > [log in to unmask] > > > > >