JISCMail - EVIDENCE-BASED-HEALTH Archives

Interesting discussion Susan……thank you.

I’d rather include the SR evidence and down grade it on quality assessment and judging my confidence in its findings. You, sounds like, will not even consider it. Perhaps you’d also not consider a study as randomized and exclude it if allocation concealment is not clearly reported.

In end, with your attempt/intent to be comprehensive, you are actually being selective. I don’t think I have a clear answer….but I like to include as much as resources allow and then explore how my appraisal explains heterogeneity in findings.

From: [log in to unmask] [mailto:[log in to unmask]] On Behalf Of Susan Fowler
Sent: Wednesday, November 19, 2014 10:27 AM
To: Ansari, Mohammed
Cc: [log in to unmask]
Subject: Re: How do you define a systematic review, meta-analysis and IPD?

If they have not searched grey lit resources appropriate to their topic and contacted experts in the field then it is not a systematic review and has no place on any continuum defining the quality of a systematic review. Additionally, they should have searched at least three medical literature databases since medline only indexes a third of peer reviewed published research articles. It is unfortunate that the medical literature is inundated with junk science and that fact that it is forces us to waste time wading through it all which impedes our ability to practice great science.

Susan

On Wed, Nov 19, 2014 at 9:09 AM, Ansari, Mohammed <[log in to unmask]> wrote:

Cannot agree or disagree. As I said “Qualitatively each one of these is on a continuum ---sometimes so poor that some may consider them not to be what they claim to be. So actually it’s a matter of where one places their defining cut-off”

Given your logic….if researchers have not searched grey literature, contacted regulatory authorities, Industry, Authors, ethics boards and courts of Law, then also they would not have searched “ALL” evidence

From: [log in to unmask] [mailto:[log in to unmask]] On Behalf Of Susan Fowler
Sent: Wednesday, November 19, 2014 10:06 AM
To: Ansari, Mohammed
Cc: [log in to unmask]
Subject: Re: How do you define a systematic review, meta-analysis and IPD?

If a "systematic review" reports only searching medline, it is NOT a systematic review because the researchers have not collected ALL the studies that relate to their question.

Susan Fowler, MLIS
Medical Librarian
Coordinator, Systematic Review Services

Evidence at Becker:
http://beckerguides.wustl.edu/ebm

Systematic Reviews Guide:
http://beckerguides.wustl.edu/SystematicReviews

Becker Medical Library, Washington University in St. Louis
314-362-8092
[log in to unmask]

On Wed, Nov 19, 2014 at 8:25 AM, Ansari, Mohammed <[log in to unmask]> wrote:

Hi Tom,

My understanding is only mine based on experience – I pen them down not looking at the definitions you provide. Qualitatively each one of these is on a continuum ---sometimes so poor that some may consider them not to be what they claim to be. So actually it’s a matter of where one places their defining cut-off. Like sensitivity and specificity there will be trade-offs. When we screen in SRs at abstract level we consider the following criteria:

At least reports:

A research question/purpose

Searching Medline

Last search date

And attempts to answer the question based on primary literature

At full text level:

Implicitly or explicitly indicate screening the search output against some eligibility criteria

Implicitly or explicitly appraise the validity and applicability of studies (separating RCT from observational studies counts as some modicum of appraisal)

Synthesizes results to reach a bottom line – not just descriptive paragraph by paragraph summary of what X, Y and Z et al…

Uses assessment of critical appraisal in synthesizing results to provide the best available synthesis

Meta-analysis:

Statistical pooling of study summary data

IPD:

Statistical pooling of raw data from more than one study

From: Evidence based health (EBH) [mailto:[log in to unmask]] On Behalf Of Tom Jefferson
Sent: Wednesday, November 19, 2014 6:25 AM
To: [log in to unmask]
Subject: How do you define a systematic review, meta-analysis and IPD?

Dear list members, I would be grateful if you could give me your views on the definitions of what constitutes a systematic review, meta-analysis and IPD. I have reported below definitions from the most popular sources of information (highest Google algorithm position). I have numbered each definition 1 to 6 for ease of commenting.
Thanks for your time.

Systematic review - Wikipedia:

1. A systematic review (also systematic literature review or structured literature review, SLR) is a literature review focused on a research question that tries to identify, appraise, select and synthesize all high quality research evidence relevant to that question

and

A systematic review aims to provide an exhaustive summary of current literature relevant to a research question. The first step of a systematic review is a thorough search of the literature for relevant papers. The Methodology section of the review will list the databases and citation indexes searched, such as Web of Science, Embase, and PubMed, as well as any hand searched individual journals. Next, the titles and the abstracts of the identified articles are checked against pre-determined criteria for eligibility and relevance. This list will always depend on the research problem. Each included study may be assigned an objective assessment of methodological quality preferably using a method conforming to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) statement (the current guideline)^[5] or the high quality standards of Cochrane collaboration.^[6]

(https://en.wikipedia.org/wiki/Systematic_review)

Systematic review - Cochrane/PRISMA:

2. A systematic review is a review of a clearly formulated question that uses systematic and explicit methods to identify, select, and critically appraise relevant research, and to collect and analyze data from the studies that are included in the review. Statistical methods (meta-analysis) may or may not be used to analyze and summarize the results of the included studies. Meta-analysis refers to the use of statistical techniques in a systematic review to integrate the results of included studies.

(http://www.cochrane.org/glossary/5#letters)

Meta-analysis - Wikipedia

3. In statistics, meta-analysis comprises statistical methods for contrasting and combining results from different studies in the hope of identifying patterns among study results, sources of disagreement among those results, or other interesting relationships that may come to light in the context of multiple studies.^[1] Meta-analysis can be thought of as "conducting research about previous research." In its simplest form, meta-analysis is done by identifying a common statistical measure that is shared between studies, such as effect size or p-value, and calculating a weighted average of that common measure. This weighting is usually related to the sample sizes of the individual studies, although it can also include other factors, such as study quality.

(https://en.wikipedia.org/wiki/Meta-analysis)

Meta-analysis - Cochrane

4. The use of statistical techniques in a systematic review to integrate the results of included studies. Sometimes misused as a synonym for systematic reviews, where the review includes a meta-analysis.

(http://www.cochrane.org/glossary/5#letterm)

Individual patient data (IPD) - Cochrane

5. Individual patient data [In meta-analysis:] The availability of raw data for each study participant in each included study, as opposed to aggregate data (summary data for the comparison groups in each study). Reviews using individual patient data require collaboration of the investigators who conducted the original studies, who must provide the necessary data.

(http://www.cochrane.org/glossary/5#letteri)

Individual patient data (IPD) - Bandolier

6. In systematic reviews this term refers to the availability of raw data for each study participant in each included trial, as opposed to aggregate data (summary data for the comparison groups in each study). Reviews using individual patient data require collaboration of the investigators who conducted the original trials, who must provide the necessary data.

(http://www.medicine.ox.ac.uk/bandolier/booth/glossary/individual.html)

Dr Tom Jefferson
Medico Chirurgo
GMC # 2527527

Confidentiality Statement - The contents of this e-mail, including its attachment, are intended for the exclusive use of the recipient and may contain confidential or privileged information. If you are not the intended recipient, you are strictly prohibited from reading, using, disclosing, copying, or distributing this e-mail or any of its contents. If you received this e-mail in error, please notify the sender by reply e-mail immediately or the Privacy Office ([log in to unmask]) and permanently delete this e-mail and its attachments, along with any copies thereof. Thank you.

Avis de confidentialité – Ce courriel, y compris ses pièces jointes, s’adresse au destinataire uniquement et pourrait contenir des renseignements confidentiels. Si vous n’êtes pas le bon destinataire, il est strictement interdit de lire, d’utiliser, de divulguer, de copier ou de diffuser ce courriel ou son contenu, en partie ou en entier. Si vous avez reçu ce courriel par erreur, veuillez en informer immédiatement l’expéditeur ou le bureau de la Protection des renseignements personnels ([log in to unmask]), puis effacez le courriel ainsi que les pièces jointes et toute autre copie. Merci.

Avis de confidentialité – Ce courriel, y compris ses pièces jointes, s’adresse au destinataire uniquement et pourrait contenir des renseignements confidentiels. Si vous n�êtes pas le bon destinataire, il est strictement interdit de lire, d’utiliser, de divulguer, de copier ou de diffuser ce courriel ou son contenu, en partie ou en entier. Si vous avez reçu ce courriel par erreur, veuillez en informer immédiatement l’expéditeur ou le bureau de la Protection des renseignements personnels ([log in to unmask]), puis effacez le courriel ainsi que les pièces jointes et toute autre copie. Merci.