I agree with Nik. You can use interval Likelihood Ratios instead of Sensitivity and Specificity with one for a positive, intermediate (indeterminate) and negative test.
Dan
Dan Mayer, MD
Professor of Emergency Medicine
Albany Medical College
Mail Code 34
47 New Scotland Ave.
Albany, NY, 12208
phone: 518-262-6180
FAX: 518-262-5029
Mobile: 518-461-3191
-----Original Message-----
From: Evidence based health (EBH) [mailto:[log in to unmask]] On Behalf Of Makretsov, Nikita [PH]
Sent: Monday, March 11, 2013 12:02 PM
To: [log in to unmask]
Subject: Re: Diagnostic test with more than two possible results
Hi Piersante,
Thank you for posting this problem. Your question is important as it is most often ignored by those practicing lab medicine (including myself, before I started learning EMB). We deal with the indeterminate results on a daily basis. I found that quite often multi-level tests are artificially binarized after the intermediate results are tossed out, and this results in spuriously high (and surely biased) sens and spec of the test, which is not the case in reality.
I would use sens/spec for the indeterminate results the way they are (50% or more or less, inc 95%CIs) along with their LRs calculated accordingly, and would then give all results for all three levels of the test.
The accuracy of two essays could be assessed by modeling their expected end results as a decision tree for all levels of the test- i.e what is trade-off between sens/spec for the patients (i.e end benefit or harm as a result of testing) at each level of the test for essay A against the essay B etc.
Pooled estimates of sens/spec are harshly criticized in the recent Cochrane RevMan software for diagnostic accuracy systematic reviews, as they often could be misleading, unless methodologic quality of the included studies is superb (which is commonly NOT the case in lab test studies). You are absolutely right binary sens/specs are hardly applicable to multi-level tests.
I like very much the way how the problem is described by Hunink M and Glasziou P in Decision Making in Health and Medicine. Also, CAT maker calculator by CEBM in Oxford has an option for multi-level diagnostic tests which I use often(www.cebm.net then check for tools).
Best regards,
Nik Vancouver,
-----Original Message-----
From: Evidence based health (EBH) [mailto:[log in to unmask]] On Behalf Of Piersante Sestini
Sent: Sunday, March 10, 2013 3:21 PM
To: [log in to unmask]
Subject: Diagnostic test with more than two possible results
One of my students is doing a systematic review about the diagnostic accuracy of interferon-gamma release assays in tuberculosis. These tests have three possible results: positive, negative, or indeterminate (this last is when there is no response to TB peptides, but also no response to a non-specific mitogen, so one cannot say whether the negative response is due to lack of sensitization or inability to respond).
All the meta-analysis software for diagnostic tests that I have found seems to be designed only for tests with binary responses (positive/negative). It is not difficult to trick them to get pooled estimates for sensitivity, specificity, and percent of indeterminate (whatever one want to call it: indeterminativity? ;-) as well as positive, negative and indeterminate Likelihood Ratios.
However, when it comes to statistical tests (for example, to compare the accuracy of two different commercial assays), they all use metaregression, that works on Odds Ratios. Of course, one cannot compute an odds ratio with three outcomes.
What is the usual solution here?
thanks,
Piersante Sestini
-----------------------------------------
CONFIDENTIALITY NOTICE: This email and any attachments may contain
confidential information that is protected by law and is for the
sole use of the individuals or entities to which it is addressed.
If you are not the intended recipient, please notify the sender by
replying to this email and destroying all copies of the
communication and attachments. Further use, disclosure, copying,
distribution of, or reliance upon the contents of this email and
attachments is strictly prohibited. To contact Albany Medical
Center, or for a copy of our privacy practices, please visit us on
the Internet at www.amc.edu.
|