Hi Piersante,
Thank you for posting this problem. Your question is important as it is most often ignored by those practicing lab medicine (including myself, before I started learning EMB). We deal with the indeterminate results on a daily basis. I found that quite often multi-level tests are artificially binarized after the intermediate results are tossed out, and this results in spuriously high (and surely biased) sens and spec of the test, which is not the case in reality.
I would use sens/spec for the indeterminate results the way they are (50% or more or less, inc 95%CIs) along with their LRs calculated accordingly, and would then give all results for all three levels of the test.
The accuracy of two essays could be assessed by modeling their expected end results as a decision tree for all levels of the test- i.e what is trade-off between sens/spec for the patients (i.e end benefit or harm as a result of testing) at each level of the test for essay A against the essay B etc.
Pooled estimates of sens/spec are harshly criticized in the recent Cochrane RevMan software for diagnostic accuracy systematic reviews, as they often could be misleading, unless methodologic quality of the included studies is superb (which is commonly NOT the case in lab test studies). You are absolutely right binary sens/specs are hardly applicable to multi-level tests.
I like very much the way how the problem is described by Hunink M and Glasziou P in Decision Making in Health and Medicine. Also, CAT maker calculator by CEBM in Oxford has an option for multi-level diagnostic tests which I use often(www.cebm.net then check for tools).
Best regards,
Nik Vancouver,
-----Original Message-----
From: Evidence based health (EBH) [mailto:[log in to unmask]] On Behalf Of Piersante Sestini
Sent: Sunday, March 10, 2013 3:21 PM
To: [log in to unmask]
Subject: Diagnostic test with more than two possible results
One of my students is doing a systematic review about the diagnostic
accuracy of interferon-gamma release assays in tuberculosis. These tests
have three possible results: positive, negative, or indeterminate (this
last is when there is no response to TB peptides, but also no response
to a non-specific mitogen, so one cannot say whether the negative
response is due to lack of sensitization or inability to respond).
All the meta-analysis software for diagnostic tests that I have found
seems to be designed only for tests with binary responses
(positive/negative). It is not difficult to trick them to get pooled
estimates for sensitivity, specificity, and percent of indeterminate
(whatever one want to call it: indeterminativity? ;-) as well as
positive, negative and indeterminate Likelihood Ratios.
However, when it comes to statistical tests (for example, to compare the
accuracy of two different commercial assays), they all use
metaregression, that works on Odds Ratios. Of course, one cannot compute
an odds ratio with three outcomes.
What is the usual solution here?
thanks,
Piersante Sestini
|