To answer Rod Jackson's question, yes a very elegant but limited method of
meta-analysis of likelihood ratios for diagnostic tests has been worked out
and published. However, it is only a partial solution. There are two types
of meta-analytic studies. The simplest type seeks to summarize several
diagnostic studies to get an overall or average expected sensitivity and
specificity (LR = sens./(1-spec.)), from which predictive values can be
calculated for different prevalence populations. These summary statistics
can be gotten by plotting the sens./spec. values from the different studies
in ROC space, and by means of a logistic transformation (log odds ratio)
plotting a summary ROC (SROC) curve. This is known as the Littenberg-Moses
method (Littenberg MD, Moses LE, Med Decis Making 1993, 13:313-321; Moses LE
et al., Stat Med 1993, 12:1293-1316). This is a very useful method;
however, it has the one drawback that it only considers between-study
variability and thus underestimates the confidence limits for the SROC
curve. This is not a big problem if the studies are large (say 100 patients
or more), but for small studies the underestimate can be considerable.
These same authors also attempted to solve the second, more difficult
meta-analysis problem of comparing a set of studies on one diagnostic
technology versus a set of studies on another. They were less successful
with this problem for two reasons. First, as mentioned above, they did not
incorporate within-study variability. The second problem is a conceptual
one rather than a strictly statistical one. Littenberg & Moses took the
points for each technology in SROC space, combined them separately to get
two separate means, and then ran a t-test. This takes points in
2-dimensional space and collapses them into 1-dimensional space. It is a
reasonable approximation if the points are very close together along both
SROC curves (meaning all the studies had similar LRs because they used
similar test thresholds). This situation can arise in the case of
standardized laboratory tests all attempting to use the same nominal
threshold to get similar LRs; however, in more subjective tests, such as
radiology or histology interpretations, this is far from the case. Because
of the different thresholds (intentional or unintentional) one has a
scatter-plot of LRs in 2-D space and a calculated regression curve.
The regression is a curve in SROC space, but is dealt with mathematically
transformed into a straight line in logistic space. Now imagine two
straight regression lines with possibly different slopes. Are the lines
statistically different from each other? Well, if they are parallel with
the same slope there are methods for determining if the distance between
them is statistically significant. There are also methods to determine if
the difference between their slopes is statistically significant. But if it
is, then the lines cross somewhere. In this case the question of whether
the lines are statistically different from each other no longer makes sense.
At the point where they cross they are not different by definition, but as
one moves away from this point they become further apart. That means with
one set of thresholds the two tests give identical results, but at other
thresholds they give wildly different results.
It certainly helps to use the above methods to put the data in an SROC plot
and look at it. But what one makes of the comparison then is more of a
medical decision (or cost-effectiveness decision) than a statistical
decision. The main thing is the plot tells one what thresholds are useful
for different purposes (i.e., to maximize sensitivity or minimize false
positives).
By the way, pooling the data from different studies, as suggested by
someone, has been shown to underestimate sensitivity and specificity (Irwig
L, et al., J Clin Epidemiol 1995, 48(1):119-30; discussion 131-2; Irwig L,
et al., Ann Intern Med 1994, 120(8):667-76).
I hope this information is helpful. These are difficult problems that have
not been given adequate attention.
David L. Doggett, Ph.D., Medical Research Analyst
Health Technology Assessment and Information Service
ECRI, a non-profit health services research organization
5200 Butler Pike, Plymouth Meeting, PA 19462 USA
(610) 825-6000 ext. 5509, FAX (610) 834-1275
[log in to unmask]
> -----Original Message-----
> From: Rod Jackson [SMTP:[log in to unmask]]
> Sent: Wednesday, March 10, 1999 11:54 PM
> To: [log in to unmask]
> Subject: HELP please: can we combine LRs from screening studies?
>
> I have a surgical colleague who has just completed a systemmatic review of
> diagnostic studies on staging of rectal cancer. He has calculated a whole
> list of likelihood ratios - interesting not one of the studies presented
> the findings using likelihood ratios. My question - is there a
> statistical
> method for combining LRs from different studies as there is for combining
> ORs or RRs from RCTs?
>
> Rod Jackson
>
> Dr Rodney Jackson MBChB PhD FAFPHM
> Associate Professor of Epidemiology
> Head of Department
> Dpt of Community Health, School of Medicine
> University of Auckland
> (Grafton Mews, 52-54 Grafton Rd)
> Private Bag 92019, Auckland, New Zealand
> Phone: +64 (0)9-3737599 ext 6343
> Fax: +64 (0)9-3737503
> e-mail: [log in to unmask]
>
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
|