Hello everyone, I've come up with the following summary of the interesting answers and discussion following my request of your opinions.

Please let me know if anyone's comments is is missing or I have misrepresetned anyone's contributions - remembering that my focus is on definitions.

Let me thank you all for the prompt and interesting comments.

 

Brassey (Chalmers)

Any review undertaken that follows a system designed to reduce systematic error (bias) and, if possible and appropriate, random error (chance).

Also suggested the following:

Last (Dictionary of Epidemiology, Fourth Edition, 2001)

SYSTEMATIC REVIEW The application of strategies that limit bias in the assembly, critical appraisal, and synthesis of all relevant studies on a specific topic. Meta-analysis may be, but is not necessarily, used

as part of this process.  

 

META-ANALYSIS The statistical synthesis of the data from separate

but similar, i.e. comparable studies

Whyte

Systematic means using a system which is intellectually sound, transparent, unambiguous and reproducible to achieve specific goals.

 

In a systematic review the system is clearly described and the goals are explicit (chiefly clarity, lack of bias and accuracy).

 

A Cochrane systematic review uses the Cochrane system, which is clearly accessible on the Cochrane website or that of its Review Groups.

 

Meta-analysis is a statistical technique which analyzes or combines (or decides not to combine) separate reports which by design and  objective address the same question (statisticians will improve on this). My point is that the term should not be used to characterize the study methodology; it makes no more sense to call a systematic review ‘a meta-analysis’ than it does to call a randomized trial ‘a t-test’. You make the same point yourself, I think.

 

Ansary

Qualitatively each one of these [terms] is on a continuum ---sometimes so poor that some may consider them not to be what they claim to be. So actually it’s a matter of where one places their defining cut-off. Like sensitivity and specificity there will be trade-offs. When we screen in SRs at abstract level we consider the following criteria:

 

At least reports:

A research question/purpose

Searching Medline

Last search date

And attempts to answer the question based on primary literature

At full text level:

Implicitly or explicitly indicate screening the search output against some eligibility criteria

Implicitly or explicitly appraise the validity and applicability of studies (separating RCT from observational studies counts as some modicum of appraisal)    

Synthesizes results to reach a bottom line – not just descriptive paragraph by paragraph summary of what X, Y and Z et al…

Uses assessment of critical appraisal in synthesizing results to provide the best available synthesis

Meta-analysis:

Statistical pooling of study summary data

IPD:

Statistical pooling of raw data from more than one study

 

Benson

For SR, #2 seems much more accurate than Wikipedia’s.  But I’d revise it some, starting with finding synonyms for “review” and “systematic” (since you generally you don’t want to use the terms you’re defining within the definition itself).  Suggested edits:  “A systematic review is an investigation review of a clearly formulated question that uses systematicmethodical and explicit methodsprocedures to identify, select, and critically appraise relevant research, and to collect and analyze data from theany appropriate studies that may be foundare included in the review. Procedures typically include searching for research studies in one or more large, reputable databases (such as PubMed or Embase for medical questions); comparing important features of each study against a list of inclusion and/or exclusion criteria (such features may include study design, subject or medical condition attributes at baseline, details of the exposure or intervention, time factors, etc.); and critical appraisal of each study for risk of bias and potential confounding factors. Statistical methods (meta-analysis) may or may not be used to analyze and summarize the results of the included studies. Meta-analysis refers to the use of statistical techniques in a systematic review to integrate the results of included studies. Typically, published reports of systematic reviews explicitly describe the database(s) and key words used in the search; the last date searched; the study inclusion/exclusion criteria; the process for determining inclusion/exclusion, risk of bias/confounding, and data extraction; a list of apparently-relevant studies that were excluded, and the reason for each exclusion; key details of each of the included studies, if any; and a summary of the findings.”

Explanations: 

-       Changing “methods” to “procedures”:  “methodical” seemed to be the best synonym for “systematic”; but then I felt “methods” was redundant and needed to be changed as well.

-       Removing “select”:  though they are indeed separate steps in the SR process, to lay readers, “identify” and “select” will seem redundant;  if you’d already identified a study as being “relevant research,” why would you need to separately “select” it?  Alternatively, the correct wording would be “…to identify possibly relevant studies, to select those that actually are relevant, and to critically appraise…”—but that level of detail is unnecessary in the opening sentence.

-       “any appropriate studies that may be found”:  allows for the all-too-common result of not having any data to collect and analyze.  But including this last phrase also begs the question:  if SR authors always “collect and analyze” data when it’s available, then why say statistical methods “may or may not” be used in the second-to-last sentence—how does one “analyze” data without using statistical methods? 

-       “Procedures typically include…” Here is where you lay it out more fully, parallel to what you’d already done with the extra sentences on meta-analysis.  Depending on the context of the definition, this new sentence and your existing last 2 sentences, as well as my new last sentence expanding on the explicit nature of SRs, may or may not need to be included.  I.e., this allows for 2 versions of the definition:  the first line only for a brief definition, and the entire paragraph for a more descriptive one.

-       “one or more large, reputable databases (such as PubMed or Embase for medical questions)”:  I believe SRs can also be done for topics in which PubMed and Embase aren’t necessarily the best databases to use, such as social welfare services or government safety programs; this wording attempts to accommodate a broader application of SR methodology.

-       Deleting your last sentence:  by putting “(meta-analysis)” in the previous sentence, your last sentence becomes unnecessary.

For meta-analysis, #4 is much more precise, but also a bit judgmental.  Who claims to have the authority to decide when a term is “misused”?  If such misuse is common, dictionary publishers generally bow to popular usage and add the widely-accepted new usage as an alternate definition.  In this vein, the definition would read:  “The use of statistical techniques in a systematic review to integrate the results of included studies. Sometimes mis Also commonly used as a synonym forto refer to a systematic reviews, where the review that includes a meta-analysis.”

For IPD, both definitions are quite similar, but beg for a little editing:  “The availability of raw data for each study participant in each included study, as opposed to aggregate data (summary data for eachthe comparison groups within each study). Reviews using individual patient data require collaboration ofwith the investigators who conducted the original studies, who must provide the necessary data.” 

-       I’d also suggest you consider taking the word “comparison” out of the first sentence, as some readers will think of this term as excluding the “experimental” group. 

-       Also note that the second sentence isn’t really part of the definition, but is more of an editorial comment; but it may be appropriate to keep, depending on where this definition would be used.  If you do keep the second sentence, perhaps also consider changing “collaboration with” to “the review authors to collaborate with”?  (And is it always true—if the original author is unavailable or being difficult, and the data is available elsewhere [e.g., EMA/FDA submissions, appendices published with the original report], could one use those instead?  Not talking about Roche, of course… J)

 

Beckles

Personally I would say that if the search methods, inclusion/exclusion criteria, appraisal/meta-analysis methods etc. have been stated clearly in the text, then it’s a systematic review.  The fact that these elements are clearly stated, allowing the review to be appraised, is what makes the review ‘systematic’ in my view. 

Gervas

A systematic review (also systematic literature review or structured literature review,
SLR) is a literature review focused on a research question that tries to identify,
appraise, select and synthesize all high quality research evidence relevant to that
question AND PRESENT IT IN SUCH A WAY THAT HELPS CLINICIANS IN THEIR DAILY WORK

Merlin

The English Editorial Board of the Health Technology Assessment (HTA) Glossary recently updated its definition of systematic review (given at http://htaglossary.net/systematic+review) to:

Systematic Review

A synthesis that collates all empirical evidence fitting pre-specified eligibility criteria in order to answer a specific research question.

Note 1: Systematic reviews are conducted according to a pre-specified protocol. The methods used are selected with a view to minimizing bias, thus providing more reliable findings from which conclusions can be drawn and decisions made.

Note 2: Many systematic reviews contain meta-analyses. A meta-analysis is the use of statistical methods to summarize the results of independent studies.

 

Various: discussion on evidence sources - single vs multiple, standard vs topic-driven

Tom’s note: not relevant to definition of SR unless part of it as a defining variable.

 

Tom Jefferson

Rome 24 November 2014