The differentiation between meta-analysis and systematic review (SR) has
interested me for quite a while. There exists a difference as to formal
definition and common usage (which is often the case with English). Whilst I
agree that the commonly accepted (in health) distinction between an SR and
meta-analysis is that meta-analysis is a statistical technique incorporated
within a quantitative SR, I think this is a misinterpretation of the formal
definitions of the two.
The Postdam Consultation on Meta-analysis (Cook DJ, Sackett DL, Spitzer WO.
1995. Methodologic Guidelines for Systematic Reviews of Randomized
Controlled Trials in Health Care from the Potsdam Consultation on
Meta-analysis. Journal of Clinical Epidemiology, 48(1), 167-171) defined a
systematic review as "application of scientific strategies that limit bias
to the systematic assembly, critical appraisal and synthesis of all relevant
studies on a specific topic". A meta-analysis was defined as a systematic
review that incorporated statistical methods to combine and suymmarise the
results of studies. So a quantitative SR is a "meta-analysis" (although not
all quantitative reviews are meta-analyses - they must be systematically
conducted to qualify as a meta-analysis). This last definition fits with
the opinion of Glass who coined the term 'meta-analysis' and developed the
the methodology of meta-analysis in psychology. However, he seems to be
remembered only for the development of the statistical process for
determining effect sizes from studies investigating similar questions but
reporting different outcomes for continuous data.
This does seem somewhat unfair and Glass states (in Hunt M. 1997. How
science takes stock: The story of meta-analysis. New York: Russell Sage
Foundation) "the whole process from first to last, that was my
contribution". The process he articulated and termed meta-analysis sounds
remarkably similar to that of systematic review as articulated at Potsdam.
Broadly this meant formulating a research question, defining a universe of
studies to sample, collecting data from the sample of studies, evaluating
the quality of the data, synthesizing and reporting the data. This
methodology was reported in 1976 and 1977 (Glass GV. 1976. primary,
Secondary, and Meta-Analysis of Research. Educational Researcher, 5, 3-8;
Glass GV. 1977. Integrating Findings; The Meta-Analysis of Research.
Research in Education, 5, 351-379) and followed on from other attempts with
psychology during that decade to develop a valid method of quantitative
research synthesis as an alternative to narrative reviews.
Of course medicine also engaged in attempts to develop quantitative review
methodologies. However, there has seems to be a tendency to term these
meta-analyses when quite clearly they do not follow the process laid out by
Glass. It has been reported that the first meta-analysis in medicine was
Beecher (1955) (Egger M, Davey Smith G. 1997. Meta-analysis: Potentials and
promise. BMJ, 315, 1371-1374). However, whilst this was an attempt at
quantitative research synthesis, it did not define a research question,
search strategy, inclusion/exclusion criteria, or assess the included
studies methodologic quality and thus seems to have more in common with a
narrative review than a SR or meta-analysis.
The development of review methodology in medicine was never subject to the
same debate as it was in psychology, so it is very difficult to track the
origins of various review methods in health. It was suggested that there
were two methods of research synthesis.1. Methodologic analysis, which
concentrated on using methodologically superior studies, although by and
large it still resulted in a narrative summarisation. 2. Pooled analysis,
which combined data from trials with goal being to acheive statistcial
significance when the treatment and control groups were compared. (Gerbag
ZB, Horwitz RI. 1988. Resolving conflicting clinical trials: Guidelines for
meta-analysis. Journal of Clinical Epidemiology, 41(5), 503-509). However,
Gerbag & Horwitz terms both methods meta-analysis.
The origin of the term systematic review is also unclear, as is its
development as a methodology. It could be it could be considerably older
than its current use suggests and may also derive from psychology. A review
incorporating 'systematic review' in its title was first referenced in
PsycLit in 1933 (Mandel, H. 1933. Racial psychic history: a detailed
introduction and a systematic review of investigations. Leipzig: Heims), and
the term was used fairly extensively in psychology after that. I would
welcome further information on the development of systematic review as a
methodology. Authors have commonly stated the need for 'systematic methods'
being applied to review writing in both psychology and medicine (Feldman KA.
1971 Using the work of others: Some observations on reviewing and
integrating. Sociology of Education, 44(Winter), 86-102; Mulrow C. 1987. The
Medical Review Article: State of Science. Annals of Internal Medicine, 106,
485-488), and I suspect the term SR might have been coined over a dinner
table somewhere.
It is hard to say at what point meta-analysis moved into health. Mulrow
examined review articles using criteria developed for assessing research
synthesis in the social sciences, and this may have had some impact. But
meta-analysis was being used prior to this (Devine EC, Cook TD. 1983. A
meta-analytic analysis of the effects of psychoeducation interventions on
the length of postsurgical hospital stay. Nursing Research, 32(5), 267-274).
What can be safely said is that at some point in the 1980s, the methodology
of meta-analysis moved from the social sciences to medicine, although I do
not think that this is adequately acknowledged.
Personally I believe that by reducing the term meta-analysis to mean the
statistical processes associated with combining results from independent
studies, we are failing to acknowledge both the extent of Glass's true
impact and the statisticians who actually developed to the methods that are
generally used in health for combining studies (Glass's effect size analysis
is not commonly used in health which is more often concerned with
dichotomous data). A quantitative systematic review should therefore be a
called a meta-analysis.
Andrew Jull
Clinical Nurse Consultant
Auckland Hospital
Private Bag 92024
Auckland
NEW ZEALAND
Phone: +64 9 3797440
Fax:+64 9 3072818 (external)
7718 (internal)
"Experience is something you don't get until just after you need it."
> -----Original Message-----
> From: Dra. Diana Rodriguez [SMTP:[log in to unmask]]
> Sent: Thursday, 11 May 2000 22:30
> To: [log in to unmask]
> Subject: About Systematic Reviews.
>
> Dear list members:
> Please, I would like to know if some person can explain me about the real
> difference between : Systematic Reviews and Meta-Analyses.
> And, Are the Meta-Analyses exclusive for Clinical trials?.
> Thanks in advance, with my best wishes .
> From the land of the Incas.
>
> DIANA .
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
|
