At the same time, don’t we need to know whether the patient probably has or does not have the condition of interest? Yes, prevalence and other factors affect PPV and NPV, but in my opinion we need to move away from the oversimplified notion that test interpretation depends on a single factor.
You are absolutely correct. This is why we should demand that diagnostic studies ONLY present the results of Sensitivity, Specificity and Likelihood ratios.
This issue has been a serious problem for many years and it is about time that more people spoke up about it. Also, journal editors and peer reviewers should be up in arms against the practice of reporting PPV and NPV.
From: Evidence based health (EBH) [[log in to unmask]]
on behalf of Benson, Teresa [[log in to unmask]]
Sent: Friday, January 30, 2015 11:59 AM
To: [log in to unmask]
Subject: Genetic tests and Predictive validity
I’ve just started reading the literature on genetic tests, and noticing how many of them tend to focus on predictive value—that is, if a certain test accurately predicts whether a patient will or won’t get a particular phenotype (condition), the authors suggest the test should be used. But if we’re deciding whether to order the test in the first place, shouldn’t we be focused on sensitivity and specificity instead, not PPV and NPV? Predictive value is so heavily dependent on disease prevalence. For example, if I want to get tested for a disease with a 2% prevalence in people like me, I could just flip a coin and regardless of the outcome, my “Coin Flip Test” would show an NPV of 98%! So what does NPV alone really tell me, if I’m not also factoring out prevalence—which would be easier done by simply looking at sensitivity and specificity? Someone please tell me where my thinking has gone awry!
For a concrete example, look at MammaPrint, a test which reports binary results. In addition to hazard ratios, study authors often tout statistically significant differences between the probabilities of recurrence-free survival in the MammaPrint-High Risk vs. MammaPrint-Low Risk groups (essentially the test’s predictive values). In the RASTER study (N = 427), 97% of the patients with a “Low Risk” test result did not experience metastasis in the next 5 years. Sounds great, right? But when you look at Sensitivity, you see that of the 33 patients in the study who did experience metastasis, only 23 of them were classified as “High Risk” by MammaPrint, for a 70% sensitivity. If patients and clinicians are looking for a test to inform their decision about adjuvant chemotherapy for early stage breast cancer, wouldn’t the fact that the test missed 10 out of 33 cases be more important than the 97% NPV, an artifact of the extremely low 5-year prevalence of metastasis in this cohort (only 33 out of 427, or 0.7%)?
Drukker et al. A prospective evaluation of a breast cancer prognosis signature in the observational RASTER study. Int J Cancer 2013. 133(4):929-36. http://www.ncbi.nlm.nih.gov/pubmed/23371464
Retel et al. Prospective cost-effectiveness analysis of genomic profiling in breast cancer. Eur J Cancer 2013. 49:3773-9. http://www.ncbi.nlm.nih.gov/pubmed/23992641 (Provides actual true/false positive/negative results)
Thanks so much!
Teresa Benson, MA, LP
Clinical Lead, Evidence-Based Medicine
McKesson Health Solutions
18211 Yorkshire Ave
Prior Lake, MN 55372
[log in to unmask]
Confidentiality Notice: This e-mail message, including any attachments, is for the sole use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply e-mail and destroy all copies of the original message.
----------------------------------------- CONFIDENTIALITY NOTICE: This email and any attachments may contain confidential information that is protected by law and is for the sole use of the individuals or entities to which it is addressed. If you are not the intended recipient, please notify the sender by replying to this email and destroying all copies of the communication and attachments. Further use, disclosure, copying, distribution of, or reliance upon the contents of this email and attachments is strictly prohibited. To contact Albany Medical Center, or for a copy of our privacy practices, please visit us on the Internet at www.amc.edu.