A Systematic review “attempts to collate all empirical evidence that fits pre-specified eligibility criteria in order to answer a specific research question. It uses explicit, systematic methods that are selected with a view to minimizing bias, thus providing more reliable findings from which conclusions can be drawn and decisions made.” Higgins JPT, Green S (editors). Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 [updated March 2011]. The Cochrane Collaboration, 2011. Available from www.cochrane-handbook.org., Section 1.2.2
specifies attempts for good reason. Researchers should to attempt to collate all empirical evidence knowing they will not be able to but also knowing that they have to do more then search medline.
I am unaware of any systematic review that gets ALL the studies and I don't believe there is any way of telling if you've got them all or not. As we now know (via AllTrials) is that approximately 30-50% of trials are unpublished - many/most of these unpublished trials are invisible to typical SR methods.If people search multiple databases and assume they find all trials that's simply wrong and potentially dangerous. They may have found ALL published trials but that's entirely different from ALL trials.I often work through the numbers to illustrate the issue. Assume there have been 20 trials and that 40% (8 trials) have not been published. A search of Medline might find 10 of the 12 published trials and an exhaustive search of multiple databases might find the other 2.So, a Medline systematic review is based on a sample of 10/20 trials (a 50% sample) while an exhaustive search might be based on 12/20 (60% sample). I'd love to see some cost-benefit figures around this. In other words, are the extra 2 trials worth the massive increase in database search time.Best wishesjonOn Wed, Nov 19, 2014 at 3:05 PM, Susan Fowler <[log in to unmask]> wrote:If a "systematic review" reports only searching medline, it is NOT a systematic review because the researchers have not collected ALL the studies that relate to their question.--Susan Fowler, MLIS
Medical Librarian
Coordinator, Systematic Review Services
Evidence at Becker:
http://beckerguides.wustl.edu/ebm
Systematic Reviews Guide:
http://beckerguides.wustl.edu/SystematicReviews
Becker Medical Library, Washington University in St. Louis
314-362-8092
[log in to unmask]On Wed, Nov 19, 2014 at 8:25 AM, Ansari, Mohammed <[log in to unmask]> wrote:Hi Tom,
My understanding is only mine based on experience – I pen them down not looking at the definitions you provide. Qualitatively each one of these is on a continuum ---sometimes so poor that some may consider them not to be what they claim to be. So actually it’s a matter of where one places their defining cut-off. Like sensitivity and specificity there will be trade-offs. When we screen in SRs at abstract level we consider the following criteria:
At least reports:
A research question/purpose
Searching Medline
Last search date
And attempts to answer the question based on primary literature
At full text level:
Implicitly or explicitly indicate screening the search output against some eligibility criteria
Implicitly or explicitly appraise the validity and applicability of studies (separating RCT from observational studies counts as some modicum of appraisal)
Synthesizes results to reach a bottom line – not just descriptive paragraph by paragraph summary of what X, Y and Z et al…
Uses assessment of critical appraisal in synthesizing results to provide the best available synthesis
Meta-analysis:
Statistical pooling of study summary data
IPD:
Statistical pooling of raw data from more than one study
From: Evidence based health (EBH) [mailto:[log in to unmask]] On Behalf Of Tom Jefferson
Sent: Wednesday, November 19, 2014 6:25 AM
To: [log in to unmask]
Subject: How do you define a systematic review, meta-analysis and IPD?
Dear list members, I would be grateful if you could give me your views on the definitions of what constitutes a systematic review, meta-analysis and IPD. I have reported below definitions from the most popular sources of information (highest Google algorithm position). I have numbered each definition 1 to 6 for ease of commenting.
Thanks for your time.
Systematic review - Wikipedia:
1. A systematic review (also systematic literature review or structured literature review, SLR) is a literature review focused on a research question that tries to identify, appraise, select and synthesize all high quality research evidence relevant to that question
and
A systematic review aims to provide an exhaustive summary of current literature relevant to a research question. The first step of a systematic review is a thorough search of the literature for relevant papers. The Methodology section of the review will list the databases and citation indexes searched, such as Web of Science, Embase, and PubMed, as well as any hand searched individual journals. Next, the titles and the abstracts of the identified articles are checked against pre-determined criteria for eligibility and relevance. This list will always depend on the research problem. Each included study may be assigned an objective assessment of methodological quality preferably using a method conforming to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) statement (the current guideline)[5] or the high quality standards of Cochrane collaboration.[6]
(https://en.wikipedia.org/wiki/Systematic_review)Systematic review - Cochrane/PRISMA:
2. A systematic review is a review of a clearly formulated question that uses systematic and explicit methods to identify, select, and critically appraise relevant research, and to collect and analyze data from the studies that are included in the review. Statistical methods (meta-analysis) may or may not be used to analyze and summarize the results of the included studies. Meta-analysis refers to the use of statistical techniques in a systematic review to integrate the results of included studies.
(http://www.cochrane.org/glossary/5#letters)
Meta-analysis - Wikipedia
3. In statistics, meta-analysis comprises statistical methods for contrasting and combining results from different studies in the hope of identifying patterns among study results, sources of disagreement among those results, or other interesting relationships that may come to light in the context of multiple studies.[1] Meta-analysis can be thought of as "conducting research about previous research." In its simplest form, meta-analysis is done by identifying a common statistical measure that is shared between studies, such as effect size or p-value, and calculating a weighted average of that common measure. This weighting is usually related to the sample sizes of the individual studies, although it can also include other factors, such as study quality.
(https://en.wikipedia.org/wiki/Meta-analysis)
Meta-analysis - Cochrane
4. The use of statistical techniques in a systematic review to integrate the results of included studies. Sometimes misused as a synonym for systematic reviews, where the review includes a meta-analysis.
(http://www.cochrane.org/glossary/5#letterm)Individual patient data (IPD) - Cochrane
5. Individual patient data [In meta-analysis:] The availability of raw data for each study participant in each included study, as opposed to aggregate data (summary data for the comparison groups in each study). Reviews using individual patient data require collaboration of the investigators who conducted the original studies, who must provide the necessary data.
Individual patient data (IPD) - Bandolier
6. In systematic reviews this term refers to the availability of raw data for each study participant in each included trial, as opposed to aggregate data (summary data for the comparison groups in each study). Reviews using individual patient data require collaboration of the investigators who conducted the original trials, who must provide the necessary data.
(http://www.medicine.ox.ac.uk/bandolier/booth/glossary/individual.html)
--
Dr Tom Jefferson
Medico Chirurgo
GMC # 2527527
Confidentiality Statement - The contents of this e-mail, including its attachment, are intended for the exclusive use of the recipient and may contain confidential or privileged information. If you are not the intended recipient, you are strictly prohibited from reading, using, disclosing, copying, or distributing this e-mail or any of its contents. If you received this e-mail in error, please notify the sender by reply e-mail immediately or the Privacy Office ([log in to unmask]) and permanently delete this e-mail and its attachments, along with any copies thereof. Thank you.
Avis de confidentialité – Ce courriel, y compris ses pièces jointes, s’adresse au destinataire uniquement et pourrait contenir des renseignements confidentiels. Si vous n’êtes pas le bon destinataire, il est strictement interdit de lire, d’utiliser, de divulguer, de copier ou de diffuser ce courriel ou son contenu, en partie ou en entier. Si vous avez reçu ce courriel par erreur, veuillez en informer immédiatement l’expéditeur ou le bureau de la Protection des renseignements personnels ([log in to unmask]), puis effacez le courriel ainsi que les pièces jointes et toute autre copie. Merci.
--