Print

Print


Dear All,

Brian, you said:

"But another consideration is sometimes tests are used for “diagnostic”
purposes – Does the patient have or not have a certain diagnosis? – an in
these cases sensitivity, specificity, PPV*, NPV*, positive likelihood
ratio, and negative likelihood ratio (* with prevalence to put into
perspective) are clear."

​What about the screening situation​, e.g. a breast cancer screening
mammography leads to a biopsy and a pathology report: if the report is
genuinely 'borderline' e.g. the pathologist reports seeing some kind of
atypia, 'indolent changes', in-situ changes etc. (changes for which I think
there is no evidence for any net benefit of treatment; ref below) How much
clarity is there then? Is this a kind of 'no gold standard situation'?  So
the so called diagnostic tests ROC curve(s) becomes guesswork? Maybe this
shouldn't be called a diagnostic test?
Owen;
(Esserman LJ, Thompson IM, Reid B. Overdiagnosis and overtreatment in
cancer: an opportunity for improvement. JAMA. 2013 Aug 28; 310(8):797-8.)

On 1 February 2015 at 14:38, Dr Amy Price <[log in to unmask]> wrote:

> Would this also be a complication in comparative effectiveness research?
>
> Best
> Amy
>
> On 2/1/15, 7:09 AM, "k.hopayian" <[log in to unmask]> wrote:
>
> >Agreed that the notion that Sn and Sp are not contingent on the
> >population is based on the premise that they were calculated in a
> >sample(s) that included all populations proportionately. This ideal is
> >pretty hard to achieve.
> >
> >The are still useful depending on why you are doing the tests. For
> >example, they would be useful when comparing two screening programmes.
> >However, if deciding whether to investigate a symptom, such as whether to
> >perform an abdo ultrasound for the isolated system of bloating in primary
> >care, you would want to know the PPV and NPV in primary care.
> >
> >Kev Hopayian
> >
> >> On 1 Feb 2015, at 11:22, Stephen Senn <[log in to unmask]>
> wrote:
> >>
> >> There is a minority but extremely interesting view that holds that
> >>sensitivity and specificity are misleading parameters. Much of the
> >>conventional discussion of this  in terms of prior probability assumes
> >>that these are permanent features of the test unaffected by prevalence
> >>so that therefore what one has to do is put them together with
> >>prevalence to calculate PPV and NPV. Suppose that this was not the case.
> >>Given PPV and NPV you could then calculate sensitivity and specificity
> >>as a function of prevalence.
> >>
> >> See the following papers
> >>
> >> 1.   Dawid AP. Properties of  Diagnostic Data Distributions. Biometrics
> >>1976; 32: 647-658.
> >> 2.   Guggenmoos-Holzmann I, van Houwelingen HC. The (in)validity of
> >>sensitivity and specificity. Statistics in Medicine 2000; 19: 1783-1792.
> >> 3.   Miettinen OS, Caro JJ. Foundations of Medical Diagnosis - What
> >>Actually Are the Parameters Involved in Bayes Theorem. Statistics in
> >>Medicine 1994; 13: 201-209.
> >>
> >>
> >>
> >> Stephen Senn
> >> Competence Center for Methodology and Statistics
> >> Luxembourg Institute of Health
> >> 1a Rue Thomas Edison
> >> L-1445 Luxembourg
> >> Tel:(+352) 26970 894
> >> Email [log in to unmask]
> >> Follow me on twitter @stephensenn
> >>
> >>
> >>
> >>
> >> -----Original Message-----
> >> From: Evidence based health (EBH)
> >>[mailto:[log in to unmask]] On Behalf Of
> >>EVIDENCE-BASED-HEALTH automatic digest system
> >> Sent: 01 February 2015 01:00
> >> To: [log in to unmask]
> >> Subject: EVIDENCE-BASED-HEALTH Digest - 30 Jan 2015 to 31 Jan 2015
> >>(#2015-25)
> >>
> >> There are 7 messages totaling 2931 lines in this issue.
> >>
> >> Topics of the day:
> >>
> >>  1. Genetic tests and Predictive validity (7)
> >>
> >> ----------------------------------------------------------------------
> >>
> >> Date:    Sat, 31 Jan 2015 02:08:24 +0000
> >> From:    "Mayer, Dan" <[log in to unmask]>
> >> Subject: Re: Genetic tests and Predictive validity
> >>
> >> Hi Teresa,
> >>
> >>
> >>
> >> You are absolutely correct.  This is why we should demand that
> >>diagnostic studies ONLY present the results of Sensitivity, Specificity
> >>and Likelihood ratios.
> >>
> >>
> >>
> >> This issue has been a serious problem for many years and it is about
> >>time that more people spoke up about it.  Also, journal editors and peer
> >>reviewers should be up in arms against the practice of reporting PPV and
> >>NPV.
> >>
> >>
> >>
> >> Best wishes
> >>
> >> Dan
> >>
> >> ________________________________
> >> From: Evidence based health (EBH)
> >>[[log in to unmask]] on behalf of Benson, Teresa
> >>[[log in to unmask]]
> >> Sent: Friday, January 30, 2015 11:59 AM
> >> To: [log in to unmask]
> >> Subject: Genetic tests and Predictive validity
> >>
> >> I¹ve just started reading the literature on genetic tests, and noticing
> >>how many of them tend to focus on predictive value‹that is, if a certain
> >>test accurately predicts whether a patient will or won¹t get a
> >>particular phenotype (condition), the authors suggest the test should be
> >>used.  But if we¹re deciding whether to order the test in the first
> >>place, shouldn¹t we be focused on sensitivity and specificity instead,
> >>not PPV and NPV?  Predictive value is so heavily dependent on disease
> >>prevalence.  For example, if I want to get tested for a disease with a
> >>2% prevalence in people like me, I could just flip a coin and regardless
> >>of the outcome, my ³Coin Flip Test² would show an NPV of 98%!  So what
> >>does NPV alone really tell me, if I¹m not also factoring out
> >>prevalence‹which would be easier done by simply looking at sensitivity
> >>and specificity?  Someone please tell me where my thinking has gone awry!
> >> For a concrete example, look at MammaPrint, a test which reports binary
> >>results.  In addition to hazard ratios, study authors often tout
> >>statistically significant differences between the probabilities of
> >>recurrence-free survival in the MammaPrint-High Risk vs. MammaPrint-Low
> >>Risk groups (essentially the test¹s predictive values).  In the RASTER
> >>study (N = 427), 97% of the patients with a ³Low Risk² test result did
> >>not experience metastasis in the next 5 years.  Sounds great, right?
> >>But when you look at Sensitivity, you see that of the 33 patients in the
> >>study who did experience metastasis, only 23 of them were classified as
> >>³High Risk² by MammaPrint, for a 70% sensitivity.  If patients and
> >>clinicians are looking for a test to inform their decision about
> >>adjuvant chemotherapy for early stage breast cancer, wouldn¹t the fact
> >>that the test missed 10 out of 33 cases be more important than the 97%
> >>NPV, an artifact of the extremely low 5-year prevalence of metastasis in
> >>this cohort (only 33 out of 427, or  0.7%)?
> >> Drukker et al. A prospective evaluation of a breast cancer prognosis
> >>signature in the observational RASTER study. Int J Cancer 2013.
> >>133(4):929-36. http://www.ncbi.nlm.nih.gov/pubmed/23371464
> >> Retel et al. Prospective cost-effectiveness analysis of genomic
> >>profiling in breast cancer. Eur J Cancer 2013. 49:3773-9.
> >>http://www.ncbi.nlm.nih.gov/pubmed/23992641  (Provides actual true/false
> >>positive/negative results)
> >>
> >> Thanks so much!
> >>
> >> Teresa Benson, MA, LP
> >> Clinical Lead, Evidence-Based Medicine
> >> McKesson Health Solutions
> >> 18211 Yorkshire Ave
> >> Prior Lake, MN  55372
> >> [log in to unmask]<mailto:[log in to unmask]>
> >> Phone: 1-952-226-4033
> >>
> >> Confidentiality Notice: This e-mail message, including any attachments,
> >>is for the sole use of the intended recipient(s) and may contain
> >>confidential and privileged information. Any unauthorized review, use,
> >>disclosure or distribution is prohibited. If you are not the intended
> >>recipient, please contact the sender by reply e-mail and destroy all
> >>copies of the original message.
> >>
> >>
> >>
> >>
> >> -----------------------------------------
> >> CONFIDENTIALITY NOTICE: This email and any attachments may contain
> >>confidential information that is protected by law and is for the sole
> >>use of the individuals or entities to which it is addressed. If you are
> >>not the intended recipient, please notify the sender by replying to this
> >>email and destroying all copies of the communication and attachments.
> >>Further use, disclosure, copying, distribution of, or reliance upon the
> >>contents of this email and attachments is strictly prohibited. To
> >>contact Albany Medical Center, or for a copy of our privacy practices,
> >>please visit us on the Internet at www.amc.edu.
> >>
> >>
> >> ------------------------------
> >>
> >> Date:    Sat, 31 Jan 2015 06:28:59 -0000
> >> From:    Michael Power <[log in to unmask]>
> >> Subject: Re: Genetic tests and Predictive validity
> >>
> >> Hi Dan and Teresa
> >>
> >>
> >>
> >> Clinicians and patients need PPVs and NPVs.
> >>
> >>
> >>
> >> So, rather than banning them, why not show graphs of PPV/NPV against
> >>the range of clinically relevant prevalences?
> >>
> >>
> >>
> >> Michael
> >>
> >>
> >>
> >> From: Evidence based health (EBH)
> >>[mailto:[log in to unmask]] On Behalf Of Mayer, Dan
> >> Sent: 31 January 2015 02:08
> >> To: [log in to unmask]
> >> Subject: Re: Genetic tests and Predictive validity
> >>
> >>
> >>
> >> Hi Teresa,
> >>
> >>
> >>
> >> You are absolutely correct.  This is why we should demand that
> >>diagnostic studies ONLY present the results of Sensitivity, Specificity
> >>and Likelihood ratios.
> >>
> >>
> >>
> >> This issue has been a serious problem for many years and it is about
> >>time that more people spoke up about it.  Also, journal editors and peer
> >>reviewers should be up in arms against the practice of reporting PPV and
> >>NPV.
> >>
> >>
> >>
> >> Best wishes
> >>
> >>
> >> Dan
> >>
> >>  _____
> >>
> >> From: Evidence based health (EBH)
> >>[[log in to unmask]] on behalf of Benson, Teresa
> >>[[log in to unmask]]
> >> Sent: Friday, January 30, 2015 11:59 AM
> >> To: [log in to unmask]
> >> Subject: Genetic tests and Predictive validity
> >>
> >> I¹ve just started reading the literature on genetic tests, and noticing
> >>how many of them tend to focus on predictive value‹that is, if a certain
> >>test accurately predicts whether a patient will or won¹t get a
> >>particular phenotype (condition), the authors suggest the test should be
> >>used.  But if we¹re deciding whether to order the test in the first
> >>place, shouldn¹t we be focused on sensitivity and specificity instead,
> >>not PPV and NPV?  Predictive value is so heavily dependent on disease
> >>prevalence.  For example, if I want to get tested for a disease with a
> >>2% prevalence in people like me, I could just flip a coin and regardless
> >>of the outcome, my ³Coin Flip Test² would show an NPV of 98%!  So what
> >>does NPV alone really tell me, if I¹m not also factoring out
> >>prevalence‹which would be easier done by simply looking at sensitivity
> >>and specificity?  Someone please tell me where my thinking has gone awry!
> >>
> >> For a concrete example, look at MammaPrint, a test which reports binary
> >>results.  In addition to hazard ratios, study authors often tout
> >>statistically significant differences between the probabilities of
> >>recurrence-free survival in the MammaPrint-High Risk vs. MammaPrint-Low
> >>Risk groups (essentially the test¹s predictive values).  In the RASTER
> >>study (N = 427), 97% of the patients with a ³Low Risk² test result did
> >>not experience metastasis in the next 5 years.  Sounds great, right?
> >>But when you look at Sensitivity, you see that of the 33 patients in the
> >>study who did experience metastasis, only 23 of them were classified as
> >>³High Risk² by MammaPrint, for a 70% sensitivity.  If patients and
> >>clinicians are looking for a test to inform their decision about
> >>adjuvant chemotherapy for early stage breast cancer, wouldn¹t the fact
> >>that the test missed 10 out of 33 cases be more important than the 97%
> >>NPV, an artifact of the extremely low 5-year prevalence of metastasis in
> >>this cohort (only 33 out of 427, or  0.7%)?
> >>
> >> Drukker et al. A prospective evaluation of a breast cancer prognosis
> >>signature in the observational RASTER study. Int J Cancer 2013.
> >>133(4):929-36. http://www.ncbi.nlm.nih.gov/pubmed/23371464
> >>
> >> Retel et al. Prospective cost-effectiveness analysis of genomic
> >>profiling in breast cancer. Eur J Cancer 2013. 49:3773-9.
> >>http://www.ncbi.nlm.nih.gov/pubmed/23992641  (Provides actual true/false
> >>positive/negative results)
> >>
> >>
> >>
> >> Thanks so much!
> >>
> >>
> >>
> >> Teresa Benson, MA, LP
> >>
> >> Clinical Lead, Evidence-Based Medicine
> >>
> >> McKesson Health Solutions
> >> 18211 Yorkshire Ave
> >>
> >> Prior Lake, MN  55372
> >>
> >> [log in to unmask] <mailto:[log in to unmask]>
> >> Phone: 1-952-226-4033
> >>
> >>
> >>
> >> Confidentiality Notice: This e-mail message, including any attachments,
> >>is for the sole use of the intended recipient(s) and may contain
> >>confidential and privileged information. Any unauthorized review, use,
> >>disclosure or distribution is prohibited. If you are not the intended
> >>recipient, please contact the sender by reply e-mail and destroy all
> >>copies of the original message.
> >>
> >>
> >>
> >>
> >>
> >>  _____
> >>
> >> ----------------------------------------- CONFIDENTIALITY NOTICE: This
> >>email and any attachments may contain confidential information that is
> >>protected by law and is for the sole use of the individuals or entities
> >>to which it is addressed. If you are not the intended recipient, please
> >>notify the sender by replying to this email and destroying all copies of
> >>the communication and attachments. Further use, disclosure, copying,
> >>distribution of, or reliance upon the contents of this email and
> >>attachments is strictly prohibited. To contact Albany Medical Center, or
> >>for a copy of our privacy practices, please visit us on the Internet at
> >>www.amc.edu.
> >>
> >> ------------------------------
> >>
> >> Date:    Sat, 31 Jan 2015 09:35:31 -0200
> >> From:    Moacyr Roberto Cuce Nobre <[log in to unmask]>
> >> Subject: Re: Genetic tests and Predictive validity
> >>
> >>
> >> I disagree Dan, if the NPV is 100% the sensitivity is 100% also,
> >>mutatis mutandis, if the PPV is 100% the specificity is 100%.
> >>
> >>
> >> I think that the problem is to present the results of accuracy, since
> >>this measure interest those involved with the test procedure. What's
> >>matter the clinician is a positive or negative test result.
> >>
> >>
> >> Cheers
> >>
> >> --
> >> Moacyr
> >>
> >> _______________________________________
> >> Moacyr Roberto Cuce Nobre, MD, MS, PhD.
> >> Diretor da Unidade de Epidemiologia Clínica Instituto do Coração
> >>(InCor) Hospital das Clínicas Faculdade de Medicina da Universidade de
> >>São Paulo
> >> 55 11 2661 5941 (fone/fax)
> >> 55 11 9133 1009 (celular)
> >>
> >> ----- Mensagem original -----
> >>
> >>
> >> De: "Dan Mayer" <[log in to unmask]>
> >> Para: [log in to unmask]
> >> Enviadas: Sábado, 31 de Janeiro de 2015 0:08:24
> >> Assunto: Re: Genetic tests and Predictive validity
> >>
> >>
> >>
> >> Hi Teresa,
> >>
> >> You are absolutely correct. This is why we should demand that
> >>diagnostic studies ONLY present the results of Sensitivity, Specificity
> >>and Likelihood ratios.
> >>
> >> This issue has been a serious problem for many years and it is about
> >>time that more people spoke up about it. Also, journal editors and peer
> >>reviewers should be up in arms against the practice of reporting PPV and
> >>NPV.
> >>
> >> Best wishes
> >>
> >> Dan
> >>
> >>
> >> From: Evidence based health (EBH)
> >>[[log in to unmask]] on behalf of Benson, Teresa
> >>[[log in to unmask]]
> >> Sent: Friday, January 30, 2015 11:59 AM
> >> To: [log in to unmask]
> >> Subject: Genetic tests and Predictive validity
> >>
> >>
> >>
> >>
> >>
> >> I¹ve just started reading the literature on genetic tests, and noticing
> >>how many of them tend to focus on predictive value‹that is, if a certain
> >>test accurately predicts whether a patient will or won¹t get a
> >>particular phenotype (condition), the authors suggest the test should be
> >>used. But if we¹re deciding whether to order the test in the first
> >>place, shouldn¹t we be focused on sensitivity and specificity instead,
> >>not PPV and NPV? Predictive value is so heavily dependent on disease
> >>prevalence. For example, if I want to get tested for a disease with a 2%
> >>prevalence in people like me, I could just flip a coin and regardless of
> >>the outcome, my ³Coin Flip Test² would show an NPV of 98%! So what does
> >>NPV alone really tell me, if I¹m not also factoring out prevalence‹which
> >>would be easier done by simply looking at sensitivity and specificity?
> >>Someone please tell me where my thinking has gone awry!
> >> For a concrete example, look at MammaPrint, a test which reports binary
> >>results. In addition to hazard ratios, study authors often tout
> >>statistically significant differences between the probabilities of
> >>recurrence-free survival in the MammaPrint-High Risk vs. MammaPrint-Low
> >>Risk groups (essentially the test¹s predictive values). In the RASTER
> >>study (N = 427), 97% of the patients with a ³Low Risk² test result did
> >>not experience metastasis in the next 5 years. Sounds great, right? But
> >>when you look at Sensitivity, you see that of the 33 patients in the
> >>study who did experience metastasis, only 23 of them were classified as
> >>³High Risk² by MammaPrint, for a 70% sensitivity. If patients and
> >>clinicians are looking for a test to inform their decision about
> >>adjuvant chemotherapy for early stage breast cancer, wouldn¹t the fact
> >>that the test missed 10 out of 33 cases be more important than the 97%
> >>NPV, an artifact of the extremely low 5-year prevalence of metastasis in
> >>this cohort (only 33 out of 427, or 0.7%)?
> >> Drukker et al. A prospective evaluation of a breast cancer prognosis
> >>signature in the observational RASTER study. Int J Cancer 2013.
> >>133(4):929-36. http://www.ncbi.nlm.nih.gov/pubmed/23371464
> >> Retel et al. Prospective cost-effectiveness analysis of genomic
> >>profiling in breast cancer. Eur J Cancer 2013. 49:3773-9.
> >>http://www.ncbi.nlm.nih.gov/pubmed/23992641 (Provides actual true/false
> >>positive/negative results)
> >>
> >> Thanks so much!
> >>
> >> Teresa Benson, MA, LP
> >> Clinical Lead, Evidence-Based Medicine
> >> McKesson Health Solutions
> >> 18211 Yorkshire Ave
> >> Prior Lake, MN 55372
> >> [log in to unmask]
> >> Phone: 1-952-226-4033
> >>
> >> Confidentiality Notice: This e-mail message, including any attachments,
> >>is for the sole use of the intended recipient(s) and may contain
> >>confidential and privileged information. Any unauthorized review, use,
> >>disclosure or distribution is prohibited. If you are not the intended
> >>recipient, please contact the sender by reply e-mail and destroy all
> >>copies of the original message.
> >>
> >>
> >>
> >>
> >> ----------------------------------------- CONFIDENTIALITY NOTICE: This
> >>email and any attachments may contain confidential information that is
> >>protected by law and is for the sole use of the individuals or entities
> >>to which it is addressed. If you are not the intended recipient, please
> >>notify the sender by replying to this email and destroying all copies of
> >>the communication and attachments. Further use, disclosure, copying,
> >>distribution of, or reliance upon the contents of this email and
> >>attachments is strictly prohibited. To contact Albany Medical Center, or
> >>for a copy of our privacy practices, please visit us on the Internet at
> >>www.amc.edu.
> >>
> >>
> >>
> >> ------------------------------
> >>
> >> Date:    Sat, 31 Jan 2015 11:39:12 +0000
> >> From:    "Huw Llewelyn [hul2]" <[log in to unmask]>
> >> Subject: Re: Genetic tests and Predictive validity
> >>
> >> Hi Theresa, Dan, Michael , Moacyr and everyone It is not only a medical
> >>finding¹s PPV (positive predictive value) that may change in another
> >>population with a different prevalence of target disease.  The
> >>specificity will also change with the prevalence.  For example, the
> >>specificity may be lower if another population contains diseases that
> >>can also ¹cause¹ the finding and the specificity may be higher (and much
> >>higher) if the population contains more people without the finding or
> >>the target disease.  The sensitivity may also change if the severity of
> >>the target disease is different in a different population.  It is also
> >>possible for a test result to have a likelihood ratio of Œone¹ and still
> >>be very powerful when used in combination with another finding to make a
> >>prediction.
> >> So it is a fallacy to assume that sensitivity and specificity are
> >>constant in different populations ­ they vary as much as PPVs.  In any
> >>case, provided one knows the prevalence, sensitivity and PPV for finding
> >>and diagnosis in a population, it is a simple matter to calculate the
> >>specificity, the NPP, etc.  However, these should not be applied to
> >>another population without checking that they are the same.  In
> >>addition, experienced doctors usually interpret intuitively the actual
> >>value of an individual patient¹s observation e.g. BP of 196/132 and do
> >>not divide it into Œnegative¹ and Œpositive¹ ranges.  Under these
> >>circumstances, there is no such thing as Œspecificity¹.  I explain this
> >>in detail in the final chapter of the 3rd edition of Oxford Handbook of
> >>Clinical Diagnosis (see also abstract #13 in
> >>http://preventingoverdiagnosis.net/documents/POD-Abstracts.docx).
> >> Huw
> >>
> >>
> >>
> >> ________________________________
> >> From: Evidence based health (EBH)
> >>[[log in to unmask]] on behalf of Michael Power
> >>[[log in to unmask]]
> >> Sent: 31 January 2015 06:28
> >> To: [log in to unmask]
> >> Subject: Re: Genetic tests and Predictive validity
> >>
> >> Hi Dan and Teresa
> >>
> >> Clinicians and patients need PPVs and NPVs.
> >>
> >> So, rather than banning them, why not show graphs of PPV/NPV against
> >>the range of clinically relevant prevalences?
> >>
> >> Michael
> >>
> >> From: Evidence based health (EBH)
> >>[mailto:[log in to unmask]] On Behalf Of Mayer, Dan
> >> Sent: 31 January 2015 02:08
> >> To: [log in to unmask]
> >> Subject: Re: Genetic tests and Predictive validity
> >>
> >>
> >> Hi Teresa,
> >>
> >>
> >>
> >> You are absolutely correct.  This is why we should demand that
> >>diagnostic studies ONLY present the results of Sensitivity, Specificity
> >>and Likelihood ratios.
> >>
> >>
> >>
> >> This issue has been a serious problem for many years and it is about
> >>time that more people spoke up about it.  Also, journal editors and peer
> >>reviewers should be up in arms against the practice of reporting PPV and
> >>NPV.
> >>
> >>
> >>
> >> Best wishes
> >>
> >> Dan
> >>
> >> ________________________________
> >> From: Evidence based health (EBH)
> >>[[log in to unmask]] on behalf of Benson, Teresa
> >>[[log in to unmask]]
> >> Sent: Friday, January 30, 2015 11:59 AM
> >> To:
> >>[log in to unmask]<mailto:
> EVIDENCE-BASED-HEALTH@JISCMAI
> >>L.AC.UK>
> >> Subject: Genetic tests and Predictive validity I¹ve just started
> >>reading the literature on genetic tests, and noticing how many of them
> >>tend to focus on predictive value‹that is, if a certain test accurately
> >>predicts whether a patient will or won¹t get a particular phenotype
> >>(condition), the authors suggest the test should be used.  But if we¹re
> >>deciding whether to order the test in the first place, shouldn¹t we be
> >>focused on sensitivity and specificity instead, not PPV and NPV?
> >>Predictive value is so heavily dependent on disease prevalence.  For
> >>example, if I want to get tested for a disease with a 2% prevalence in
> >>people like me, I could just flip a coin and regardless of the outcome,
> >>my ³Coin Flip Test² would show an NPV of 98%!  So what does NPV alone
> >>really tell me, if I¹m not also factoring out prevalence‹which would be
> >>easier done by simply looking at sensitivity and specificity?  Someone
> >>please tell me where my thinking has gone awry!
> >> For a concrete example, look at MammaPrint, a test which reports binary
> >>results.  In addition to hazard ratios, study authors often tout
> >>statistically significant differences between the probabilities of
> >>recurrence-free survival in the MammaPrint-High Risk vs. MammaPrint-Low
> >>Risk groups (essentially the test¹s predictive values).  In the RASTER
> >>study (N = 427), 97% of the patients with a ³Low Risk² test result did
> >>not experience metastasis in the next 5 years.  Sounds great, right?
> >>But when you look at Sensitivity, you see that of the 33 patients in the
> >>study who did experience metastasis, only 23 of them were classified as
> >>³High Risk² by MammaPrint, for a 70% sensitivity.  If patients and
> >>clinicians are looking for a test to inform their decision about
> >>adjuvant chemotherapy for early stage breast cancer, wouldn¹t the fact
> >>that the test missed 10 out of 33 cases be more important than the 97%
> >>NPV, an artifact of the extremely low 5-year prevalence of metastasis in
> >>this cohort (only 33 out of 427, or  0.7%)?
> >> Drukker et al. A prospective evaluation of a breast cancer prognosis
> >>signature in the observational RASTER study. Int J Cancer 2013.
> >>133(4):929-36. http://www.ncbi.nlm.nih.gov/pubmed/23371464
> >> Retel et al. Prospective cost-effectiveness analysis of genomic
> >>profiling in breast cancer. Eur J Cancer 2013. 49:3773-9.
> >>http://www.ncbi.nlm.nih.gov/pubmed/23992641  (Provides actual true/false
> >>positive/negative results)
> >>
> >> Thanks so much!
> >>
> >> Teresa Benson, MA, LP
> >> Clinical Lead, Evidence-Based Medicine
> >> McKesson Health Solutions
> >> 18211 Yorkshire Ave
> >> Prior Lake, MN  55372
> >> [log in to unmask]<mailto:[log in to unmask]>
> >> Phone: 1-952-226-4033
> >>
> >> Confidentiality Notice: This e-mail message, including any attachments,
> >>is for the sole use of the intended recipient(s) and may contain
> >>confidential and privileged information. Any unauthorized review, use,
> >>disclosure or distribution is prohibited. If you are not the intended
> >>recipient, please contact the sender by reply e-mail and destroy all
> >>copies of the original message.
> >>
> >>
> >> ________________________________
> >> ----------------------------------------- CONFIDENTIALITY NOTICE: This
> >>email and any attachments may contain confidential information that is
> >>protected by law and is for the sole use of the individuals or entities
> >>to which it is addressed. If you are not the intended recipient, please
> >>notify the sender by replying to this email and destroying all copies of
> >>the communication and attachments. Further use, disclosure, copying,
> >>distribution of, or reliance upon the contents of this email and
> >>attachments is strictly prohibited. To contact Albany Medical Center, or
> >>for a copy of our privacy practices, please visit us on the Internet at
> >>www.amc.edu<http://www.amc.edu>.
> >>
> >> ------------------------------
> >>
> >> Date:    Sat, 31 Jan 2015 12:31:29 +0000
> >> From:    Brian Alper MD <[log in to unmask]>
> >> Subject: Re: Genetic tests and Predictive validity
> >>
> >> Reporting diagnostic and prognostic parameters to help clinicians and
> >>policy makers understood the use of tests is challenging.   There are
> >>many possible parameters to report and varied understanding such that
> >>one measure is not uniformly understood by all.
> >>
> >> The most sophisticated may want likelihood ratios and understand how to
> >>apply them with respect to pretest likelihood.   But many want simpler
> >>yes/no type of information.  For the simpler approach sensitivity and
> >>specificity are too often confused for rule in and rule out, and are too
> >>often confused for representing the likelihood of an answer in practice
> >>(which does not account for the pretest likelihood at all).
> >>
> >> For these reasons positive and negative predictive values appear to be
> >>the simplest yet clinically meaningful results to report.  However they
> >>are only accurate if the pretest likelihood is similar in practice to
> >>the source where they are being reported from.  It is possible to report
> >>PPV and NPV for different pretest likelihoods or different
> >>representative populations.
> >>
> >> In determining whether a test is useful, the PPV and NPV in isolation
> >>is insufficient.  Only if PPV or NPV is different enough from the
> >>pretest likelihood to cross a decision threshold (treatment threshold or
> >>testing threshold) does the test change action.  In the example below
> >>with a pretest likelihood of 2% and a coin flip resulting in PPV 2% and
> >>NPV 98% the post-test results are no different than the pretest
> >>likelihood.
> >>
> >> Other parameters (accuracy, diagnostic odds ratio) that combine
> >>sensitivity and specificity into a single measure are easier to combine
> >>and report statistically but I generally avoid these parameters because
> >>they are less informative for clinical use - in most clinical situations
> >>sensitivity and specificity are not equally important; a useful
> >>one-sided test (eg useful if positive, ignored if negative) becomes less
> >>clear when viewing it for overall accuracy.
> >>
> >>
> >> But another consideration is sometimes tests are used for "diagnostic"
> >>purposes - Does the patient have or not have a certain diagnosis? - an
> >>in these cases sensitivity, specificity, PPV*, NPV*, positive likelihood
> >>ratio, and negative likelihood ratio (* with prevalence to put into
> >>perspective) are clear.
> >>
> >> Sometimes tests are used for prognostic purposes, and may be used as a
> >>continuous measure with different predictions at different test results.
> >>  In these cases presenting the expected prevalence (or probability or
> >>likelihood) at different test results may be easier to interpret than
> >>selecting a single cutoff and converting it to
> >>sensitivity/specificity/etc.
> >>
> >>
> >> In general all of the comments above reflect diagnostic and prognostic
> >>testing, whether genetic or not.  If the test is genetic the results can
> >>usually be handled similarly if the clinical question is specific to the
> >>person being tested.   There may be some scenarios where familial
> >>implications (cross-generation interpretation) or combinatorial
> >>implications (multigene testing) make the overall reporting more
> >>challenging.
> >>
> >> Brian S. Alper, MD, MSPH, FAAFP
> >> Founder of DynaMed
> >> Vice President of EBM Research and Development, Quality & Standards
> >>dynamed.ebscohost.com
> >>
> >> From: Evidence based health (EBH)
> >>[mailto:[log in to unmask]] On Behalf Of Benson,
> Teresa
> >> Sent: Friday, January 30, 2015 12:00 PM
> >> To: [log in to unmask]
> >> Subject: Genetic tests and Predictive validity
> >>
> >> I've just started reading the literature on genetic tests, and noticing
> >>how many of them tend to focus on predictive value-that is, if a certain
> >>test accurately predicts whether a patient will or won't get a
> >>particular phenotype (condition), the authors suggest the test should be
> >>used.  But if we're deciding whether to order the test in the first
> >>place, shouldn't we be focused on sensitivity and specificity instead,
> >>not PPV and NPV?  Predictive value is so heavily dependent on disease
> >>prevalence.  For example, if I want to get tested for a disease with a
> >>2% prevalence in people like me, I could just flip a coin and regardless
> >>of the outcome, my "Coin Flip Test" would show an NPV of 98%!  So what
> >>does NPV alone really tell me, if I'm not also factoring out
> >>prevalence-which would be easier done by simply looking at sensitivity
> >>and specificity?  Someone please tell me where my thinking has gone awry!
> >> For a concrete example, look at MammaPrint, a test which reports binary
> >>results.  In addition to hazard ratios, study authors often tout
> >>statistically significant differences between the probabilities of
> >>recurrence-free survival in the MammaPrint-High Risk vs. MammaPrint-Low
> >>Risk groups (essentially the test's predictive values).  In the RASTER
> >>study (N = 427), 97% of the patients with a "Low Risk" test result did
> >>not experience metastasis in the next 5 years.  Sounds great, right?
> >>But when you look at Sensitivity, you see that of the 33 patients in the
> >>study who did experience metastasis, only 23 of them were classified as
> >>"High Risk" by MammaPrint, for a 70% sensitivity.  If patients and
> >>clinicians are looking for a test to inform their decision about
> >>adjuvant chemotherapy for early stage breast cancer, wouldn't the fact
> >>that the test missed 10 out of 33 cases be more important than the 97%
> >>NPV, an artifact of the extremely low 5-year prevalence of metastasis in
> >>this cohort (only 33 out of 427, or  0.7%)?
> >> Drukker et al. A prospective evaluation of a breast cancer prognosis
> >>signature in the observational RASTER study. Int J Cancer 2013.
> >>133(4):929-36. http://www.ncbi.nlm.nih.gov/pubmed/23371464
> >> Retel et al. Prospective cost-effectiveness analysis of genomic
> >>profiling in breast cancer. Eur J Cancer 2013. 49:3773-9.
> >>http://www.ncbi.nlm.nih.gov/pubmed/23992641  (Provides actual true/false
> >>positive/negative results)
> >>
> >> Thanks so much!
> >>
> >> Teresa Benson, MA, LP
> >> Clinical Lead, Evidence-Based Medicine
> >> McKesson Health Solutions
> >> 18211 Yorkshire Ave
> >> Prior Lake, MN  55372
> >> [log in to unmask]<mailto:[log in to unmask]>
> >> Phone: 1-952-226-4033
> >>
> >> Confidentiality Notice: This e-mail message, including any attachments,
> >>is for the sole use of the intended recipient(s) and may contain
> >>confidential and privileged information. Any unauthorized review, use,
> >>disclosure or distribution is prohibited. If you are not the intended
> >>recipient, please contact the sender by reply e-mail and destroy all
> >>copies of the original message.
> >>
> >>
> >> ------------------------------
> >>
> >> Date:    Sat, 31 Jan 2015 14:59:57 +0000
> >> From:    "Huw Llewelyn [hul2]" <[log in to unmask]>
> >> Subject: Re: Genetic tests and Predictive validity
> >>
> >> Hi Moacyr, Theresa, Dan, Michael, Brian and everyone I agree that prior
> >>probabilities are important.  However, these should not be
> >>non-evidence-based subjective guesses but probabilities (i.e. positive
> >>predictive values) of diagnoses in a traditional differential diagnostic
> >>list.  This list and the probabilities are suggested by a presenting
> >>complaint or another finding (e.g. a chest x ray appearance) with a
> >>shorter list of possibilities.  One of these diagnoses are chosen, and
> >>other findings looked for that occur commonly in that diagnosis (i.e.
> >>the sensitivity is high) and rarely or never in at least one other
> >>diagnosis in the list (i.e. the sensitivity is low or zero).  If the
> >>finding were present, this would make the probability of the chosen
> >>diagnosis higher and the probability of the other diagnosis lower or
> >>zero.  Other findings are sought to try to make all but one of the
> >>differential diagnoses improbable.
> >> In this reasoning process, which experienced doctors use when
> >>explaining their reasoning, we use ratios of sensitivities, NOT the
> >>ratio of sensitivity to false positive rate (i.e. one minus the
> >>specificity).  Differential diagnostic reasoning uses a statistical
> >>dependence assumption which safely underestimates diagnostic
> >>probabilities whereas using likelihood ratios uses a statistical
> >>independence assumption that overestimates probabilities (and thus may
> >>over-diagnose).  I explain this reasoning process in the Oxford Handbook
> >>of Clinical Diagnosis (see page 6 and 622 with the mathematical proof on
> >>page 638) in Œlook inside¹ on the Amazon web-site:
> >>
> http://www.amazon.co.uk/Handbook-Clinical-Diagnosis-Medical-Handbooks/dp/
> >>019967986X#reader_019967986X.
> >> Best wishes
> >> Huw
> >>
> >>
> >>
> >> ________________________________
> >> From: Moacyr Roberto Cuce Nobre [[log in to unmask]]
> >> Sent: 31 January 2015 12:15
> >> To: Huw Llewelyn [hul2]
> >> Subject: Re: Genetic tests and Predictive validity
> >>
> >> Hi Huw, dont you think that the output to the clinician is the pretest
> >>probability of the patient, that better reflects the individual
> >>uncertainty, rather than the prevalence as a population data.
> >>
> >> --
> >> Moacyr
> >>
> >> _______________________________________
> >> Moacyr Roberto Cuce Nobre, MD, MS, PhD.
> >> Diretor da Unidade de Epidemiologia Clínica Instituto do Coração
> >>(InCor) Hospital das Clínicas Faculdade de Medicina da Universidade de
> >>São Paulo
> >> 55 11 2661 5941 (fone/fax)
> >> 55 11 9133 1009 (celular)
> >>
> >> ________________________________
> >> De: "Huw Llewelyn [hul2]" <[log in to unmask]>
> >> Para: [log in to unmask]
> >> Enviadas: Sábado, 31 de Janeiro de 2015 9:39:12
> >> Assunto: Re: Genetic tests and Predictive validity
> >>
> >> Hi Theresa, Dan, Michael , Moacyr and everyone It is not only a medical
> >>finding¹s PPV (positive predictive value) that may change in another
> >>population with a different prevalence of target disease.  The
> >>specificity will also change with the prevalence.  For example, the
> >>specificity may be lower if another population contains diseases that
> >>can also ¹cause¹ the finding and the specificity may be higher (and much
> >>higher) if the population contains more people without the finding or
> >>the target disease.  The sensitivity may also change if the severity of
> >>the target disease is different in a different population.  It is also
> >>possible for a test result to have a likelihood ratio of Œone¹ and still
> >>be very powerful when used in combination with another finding to make a
> >>prediction.
> >> So it is a fallacy to assume that sensitivity and specificity are
> >>constant in different populations ­ they vary as much as PPVs.  In any
> >>case, provided one knows the prevalence, sensitivity and PPV for finding
> >>and diagnosis in a population, it is a simple matter to calculate the
> >>specificity, the NPP, etc.  However, these should not be applied to
> >>another population without checking that they are the same.  In
> >>addition, experienced doctors usually interpret intuitively the actual
> >>value of an individual patient¹s observation e.g. BP of 196/132 and do
> >>not divide it into Œnegative¹ and Œpositive¹ ranges.  Under these
> >>circumstances, there is no such thing as Œspecificity¹.  I explain this
> >>in detail in the final chapter of the 3rd edition of Oxford Handbook of
> >>Clinical Diagnosis (see also abstract #13 in
> >>http://preventingoverdiagnosis.net/documents/POD-Abstracts.docx).
> >> Huw
> >>
> >>
> >>
> >> ________________________________
> >> From: Evidence based health (EBH)
> >>[[log in to unmask]] on behalf of Michael Power
> >>[[log in to unmask]]
> >> Sent: 31 January 2015 06:28
> >> To: [log in to unmask]
> >> Subject: Re: Genetic tests and Predictive validity
> >>
> >> Hi Dan and Teresa
> >>
> >> Clinicians and patients need PPVs and NPVs.
> >>
> >> So, rather than banning them, why not show graphs of PPV/NPV against
> >>the range of clinically relevant prevalences?
> >>
> >> Michael
> >>
> >> From: Evidence based health (EBH)
> >>[mailto:[log in to unmask]] On Behalf Of Mayer, Dan
> >> Sent: 31 January 2015 02:08
> >> To: [log in to unmask]
> >> Subject: Re: Genetic tests and Predictive validity
> >>
> >>
> >> Hi Teresa,
> >>
> >>
> >>
> >> You are absolutely correct.  This is why we should demand that
> >>diagnostic studies ONLY present the results of Sensitivity, Specificity
> >>and Likelihood ratios.
> >>
> >>
> >>
> >> This issue has been a serious problem for many years and it is about
> >>time that more people spoke up about it.  Also, journal editors and peer
> >>reviewers should be up in arms against the practice of reporting PPV and
> >>NPV.
> >>
> >>
> >>
> >> Best wishes
> >>
> >> Dan
> >>
> >> ________________________________
> >> From: Evidence based health (EBH)
> >>[[log in to unmask]] on behalf of Benson, Teresa
> >>[[log in to unmask]]
> >> Sent: Friday, January 30, 2015 11:59 AM
> >> To:
> >>[log in to unmask]<mailto:
> EVIDENCE-BASED-HEALTH@JISCMAI
> >>L.AC.UK>
> >> Subject: Genetic tests and Predictive validity I¹ve just started
> >>reading the literature on genetic tests, and noticing how many of them
> >>tend to focus on predictive value‹that is, if a certain test accurately
> >>predicts whether a patient will or won¹t get a particular phenotype
> >>(condition), the authors suggest the test should be used.  But if we¹re
> >>deciding whether to order the test in the first place, shouldn¹t we be
> >>focused on sensitivity and specificity instead, not PPV and NPV?
> >>Predictive value is so heavily dependent on disease prevalence.  For
> >>example, if I want to get tested for a disease with a 2% prevalence in
> >>people like me, I could just flip a coin and regardless of the outcome,
> >>my ³Coin Flip Test² would show an NPV of 98%!  So what does NPV alone
> >>really tell me, if I¹m not also factoring out prevalence‹which would be
> >>easier done by simply looking at sensitivity and specificity?  Someone
> >>please tell me where my thinking has gone awry!
> >> For a concrete example, look at MammaPrint, a test which reports binary
> >>results.  In addition to hazard ratios, study authors often tout
> >>statistically significant differences between the probabilities of
> >>recurrence-free survival in the MammaPrint-High Risk vs. MammaPrint-Low
> >>Risk groups (essentially the test¹s predictive values).  In the RASTER
> >>study (N = 427), 97% of the patients with a ³Low Risk² test result did
> >>not experience metastasis in the next 5 years.  Sounds great, right?
> >>But when you look at Sensitivity, you see that of the 33 patients in the
> >>study who did experience metastasis, only 23 of them were classified as
> >>³High Risk² by MammaPrint, for a 70% sensitivity.  If patients and
> >>clinicians are looking for a test to inform their decision about
> >>adjuvant chemotherapy for early stage breast cancer, wouldn¹t the fact
> >>that the test missed 10 out of 33 cases be more important than the 97%
> >>NPV, an artifact of the extremely low 5-year prevalence of metastasis in
> >>this cohort (only 33 out of 427, or  0.7%)?
> >> Drukker et al. A prospective evaluation of a breast cancer prognosis
> >>signature in the observational RASTER study. Int J Cancer 2013.
> >>133(4):929-36. http://www.ncbi.nlm.nih.gov/pubmed/23371464
> >> Retel et al. Prospective cost-effectiveness analysis of genomic
> >>profiling in breast cancer. Eur J Cancer 2013. 49:3773-9.
> >>http://www.ncbi.nlm.nih.gov/pubmed/23992641  (Provides actual true/false
> >>positive/negative results)
> >>
> >> Thanks so much!
> >>
> >> Teresa Benson, MA, LP
> >> Clinical Lead, Evidence-Based Medicine
> >> McKesson Health Solutions
> >> 18211 Yorkshire Ave
> >> Prior Lake, MN  55372
> >> [log in to unmask]<mailto:[log in to unmask]>
> >> Phone: 1-952-226-4033
> >>
> >> Confidentiality Notice: This e-mail message, including any attachments,
> >>is for the sole use of the intended recipient(s) and may contain
> >>confidential and privileged information. Any unauthorized review, use,
> >>disclosure or distribution is prohibited. If you are not the intended
> >>recipient, please contact the sender by reply e-mail and destroy all
> >>copies of the original message.
> >>
> >>
> >> ________________________________
> >> ----------------------------------------- CONFIDENTIALITY NOTICE: This
> >>email and any attachments may contain confidential information that is
> >>protected by law and is for the sole use of the individuals or entities
> >>to which it is addressed. If you are not the intended recipient, please
> >>notify the sender by replying to this email and destroying all copies of
> >>the communication and attachments. Further use, disclosure, copying,
> >>distribution of, or reliance upon the contents of this email and
> >>attachments is strictly prohibited. To contact Albany Medical Center, or
> >>for a copy of our privacy practices, please visit us on the Internet at
> >>www.amc.edu<http://www.amc.edu>.
> >>
> >> ------------------------------
> >>
> >> Date:    Sat, 31 Jan 2015 16:45:44 +0000
> >> From:    "McCormack, James" <[log in to unmask]>
> >> Subject: Re: Genetic tests and Predictive validity
> >>
> >> Hi Everyone - here is a useful rough rule of thumb for Likelihood
> >>Ratios that a clinician can use which puts numbers to disease
> >>probability and you can see when an LR can basically rule in our out a
> >>disease.
> >>
> >> The only caveat is this is ONLY accurate if the pretest probability
> >>lies between 10% and 90% so it doesn¹t apply to low prevalent (<10%)
> >>conditions so it may not be all that applicable to this discussion.
> >>
> >> A LR of 2 increases the absolute probability of disease FROM your
> >>pre-test probability by 15% - for every additional one increase in LR
> >>you add another 5% absolute change to your probability - so a LR of 3
> >>increases your probability of disease by 15% plus 2 X 5% or a total
> >>increase of 25% over your pre-test probability.
> >>
> >> Going in the other direction
> >>
> >> A LR of 0.5 decreases the absolute probability of disease FROM your
> >>pre-test probability by 15% - for every additional 0.1 lowering in LR
> >>you add another 5% absolute change to your probability - so a LR of 0.3
> >>decreases probability by 15% plus 2 X 5% or a total decrease of 25% from
> >>your pre-test probability.
> >>
> >> The original idea for this came from this article
> >>http://onlinelibrary.wiley.com/doi/10.1046/j.1525-1497.2002.10750.x/full
> >>
> >> and we published a discussion on probabilistic reasoning in primary
> >>care here http://goo.gl/Rnl2q2
> >>
> >> Thanks
> >>
> >> James
> >>
> >>
> >>
> >> On Jan 31, 2015, at 4:31 AM, Brian Alper MD
> >><[log in to unmask]<mailto:[log in to unmask]>> wrote:
> >>
> >> Reporting diagnostic and prognostic parameters to help clinicians and
> >>policy makers understood the use of tests is challenging.   There are
> >>many possible parameters to report and varied understanding such that
> >>one measure is not uniformly understood by all.
> >>
> >> The most sophisticated may want likelihood ratios and understand how to
> >>apply them with respect to pretest likelihood.   But many want simpler
> >>yes/no type of information.  For the simpler approach sensitivity and
> >>specificity are too often confused for rule in and rule out, and are too
> >>often confused for representing the likelihood of an answer in practice
> >>(which does not account for the pretest likelihood at all).
> >>
> >> For these reasons positive and negative predictive values appear to be
> >>the simplest yet clinically meaningful results to report.  However they
> >>are only accurate if the pretest likelihood is similar in practice to
> >>the source where they are being reported from.  It is possible to report
> >>PPV and NPV for different pretest likelihoods or different
> >>representative populations.
> >>
> >> In determining whether a test is useful, the PPV and NPV in isolation
> >>is insufficient.  Only if PPV or NPV is different enough from the
> >>pretest likelihood to cross a decision threshold (treatment threshold or
> >>testing threshold) does the test change action.  In the example below
> >>with a pretest likelihood of 2% and a coin flip resulting in PPV 2% and
> >>NPV 98% the post-test results are no different than the pretest
> >>likelihood.
> >>
> >> Other parameters (accuracy, diagnostic odds ratio) that combine
> >>sensitivity and specificity into a single measure are easier to combine
> >>and report statistically but I generally avoid these parameters because
> >>they are less informative for clinical use ­ in most clinical situations
> >>sensitivity and specificity are not equally important; a useful
> >>one-sided test (eg useful if positive, ignored if negative) becomes less
> >>clear when viewing it for overall accuracy.
> >>
> >>
> >> But another consideration is sometimes tests are used for ³diagnostic²
> >>purposes ­ Does the patient have or not have a certain diagnosis? ­ an
> >>in these cases sensitivity, specificity, PPV*, NPV*, positive likelihood
> >>ratio, and negative likelihood ratio (* with prevalence to put into
> >>perspective) are clear.
> >>
> >> Sometimes tests are used for prognostic purposes, and may be used as a
> >>continuous measure with different predictions at different test results.
> >>  In these cases presenting the expected prevalence (or probability or
> >>likelihood) at different test results may be easier to interpret than
> >>selecting a single cutoff and converting it to
> >>sensitivity/specificity/etc.
> >>
> >>
> >> In general all of the comments above reflect diagnostic and prognostic
> >>testing, whether genetic or not.  If the test is genetic the results can
> >>usually be handled similarly if the clinical question is specific to the
> >>person being tested.   There may be some scenarios where familial
> >>implications (cross-generation interpretation) or combinatorial
> >>implications (multigene testing) make the overall reporting more
> >>challenging.
> >>
> >> Brian S. Alper, MD, MSPH, FAAFP
> >> Founder of DynaMed
> >> Vice President of EBM Research and Development, Quality & Standards
> >>dynamed.ebscohost.com<http://dynamed.ebscohost.com/>
> >>
> >> From: Evidence based health (EBH)
> >>[mailto:[log in to unmask]] On Behalf Of Benson,
> Teresa
> >> Sent: Friday, January 30, 2015 12:00 PM
> >> To:
> >>[log in to unmask]<mailto:
> EVIDENCE-BASED-HEALTH@JISCMAI
> >>L.AC.UK>
> >> Subject: Genetic tests and Predictive validity
> >>
> >> I¹ve just started reading the literature on genetic tests, and noticing
> >>how many of them tend to focus on predictive value‹that is, if a certain
> >>test accurately predicts whether a patient will or won¹t get a
> >>particular phenotype (condition), the authors suggest the test should be
> >>used.  But if we¹re deciding whether to order the test in the first
> >>place, shouldn¹t we be focused on sensitivity and specificity instead,
> >>not PPV and NPV?  Predictive value is so heavily dependent on disease
> >>prevalence.  For example, if I want to get tested for a disease with a
> >>2% prevalence in people like me, I could just flip a coin and regardless
> >>of the outcome, my ³Coin Flip Test² would show an NPV of 98%!  So what
> >>does NPV alone really tell me, if I¹m not also factoring out
> >>prevalence‹which would be easier done by simply looking at sensitivity
> >>and specificity?  Someone please tell me where my thinking has gone awry!
> >> For a concrete example, look at MammaPrint, a test which reports binary
> >>results.  In addition to hazard ratios, study authors often tout
> >>statistically significant differences between the probabilities of
> >>recurrence-free survival in the MammaPrint-High Risk vs. MammaPrint-Low
> >>Risk groups (essentially the test¹s predictive values).  In the RASTER
> >>study (N = 427), 97% of the patients with a ³Low Risk² test result did
> >>not experience metastasis in the next 5 years.  Sounds great, right?
> >>But when you look at Sensitivity, you see that of the 33 patients in the
> >>study who did experience metastasis, only 23 of them were classified as
> >>³High Risk² by MammaPrint, for a 70% sensitivity.  If patients and
> >>clinicians are looking for a test to inform their decision about
> >>adjuvant chemotherapy for early stage breast cancer, wouldn¹t the fact
> >>that the test missed 10 out of 33 cases be more important than the 97%
> >>NPV, an artifact of the extremely low 5-year prevalence of metastasis in
> >>this cohort (only 33 out of 427, or  0.7%)?
> >> Drukker et al. A prospective evaluation of a breast cancer prognosis
> >>signature in the observational RASTER study. Int J Cancer 2013.
> >>133(4):929-36.http://www.ncbi.nlm.nih.gov/pubmed/23371464
> >> Retel et al. Prospective cost-effectiveness analysis of genomic
> >>profiling in breast cancer. Eur J Cancer 2013. 49:3773-9.
> >>http://www.ncbi.nlm.nih.gov/pubmed/23992641  (Provides actual true/false
> >>positive/negative results)
> >>
> >> Thanks so much!
> >>
> >> Teresa Benson, MA, LP
> >> Clinical Lead, Evidence-Based Medicine
> >> McKesson Health Solutions
> >> 18211 Yorkshire Ave
> >> Prior Lake, MN  55372
> >> [log in to unmask]<mailto:[log in to unmask]>
> >> Phone: 1-952-226-4033
> >>
> >> Confidentiality Notice: This e-mail message, including any attachments,
> >>is for the sole use of the intended recipient(s) and may contain
> >>confidential and privileged information. Any unauthorized review, use,
> >>disclosure or distribution is prohibited. If you are not the intended
> >>recipient, please contact the sender by reply e-mail and destroy all
> >>copies of the original message.
> >>
> >> ------------------------------
> >>
> >> End of EVIDENCE-BASED-HEALTH Digest - 30 Jan 2015 to 31 Jan 2015
> >>(#2015-25)
> >>
> >>*************************************************************************
> >>**
>



-- 
Owen Dempsey MBBS MSc MRCGP RCGP cert II

07760 164420

GPwsi Substance Misuse Locala and Kirklees Lifeline Addiction Service