Sensitivity = TP / TP + FN
Specificity = TN / TN + FP
Accuracy = TP + TN/ TP + TN + FP + FN
With the prevalence of 1:1,000 the accuracy is very close to the specificity because the contribution of TP and FN to the Accuracy calculation in negligible (0-1/1000 of the numerator and 1/1000 of the denominator)
So consider a population of 1001 with:
TN = 950
FP = 50
TP + FN = 1
TP is either 0 or 1 if literally a person, or ranges between 0-1 if using these numbers to represent the population
Positive predictive value = TP / TP + FP
The value of positive predictive value can be anywhere between 0 and 1/51, so a range of 0-2%.
The “correct answer” in the paragraph assumes 100% sensitivity and 95% specificity. This is actually 95.005% accuracy.
The example is basically showing how high specificity and very low prevalence can still lead to very low positive predictive value.
Brian S. Alper, MD, MSPH
Editor-in-Chief, DynaMed (www.ebscohost.com/dynamed)
From: Evidence based health (EBH) [mailto:[log in to unmask]] On Behalf Of Simon Hatcher
Sent: Thursday, February 10, 2011 5:34 AM
To: [log in to unmask]
Subject: Re: Question about screening and denominators
After another night this time on the red wine my friend has provided me with the source of the problem - it is John Lanchester's book "Whoops" where Lanchester writes "if a test for some disease is 95% accurate, and the disease affects one person in a thousand, and you go for a test and it comes back positive, what's the probability that you have the disease? Most respondents say, well the test is accurate, so the probability is 95%. The correct answer is 2%, because if you test 1000 people, the test will give fifty positives, whereas only one of the population has the illness". The reference for this is Nassim Nicholas Taleb's "Fooled by Randomness". Is he right?
From: Evidence based health (EBH) on behalf of Miranda Cumpston (Med)
Sent: Thu 10/02/2011 13:08
To: [log in to unmask]
Subject: Re: Question about screening and denominators
Hi all,
Just for the sake of it, here are some numbers, assuming that you mean 95% for both specificity and sensitivity (which is a pretty unlikely test) – that is, the test will detect 95% of true positives and true negatives (using a population of 100,000 to keep everything in whole numbers):
| True + | True - | Total |
Test + | 95 | 4,995 | 5,090 |
Test - | 5 | 94,905 | 94,910 |
Total | 100 | 99,900 | 100,000 |
So, assuming you have a positive test (i.e. you’re in the top row), then the probability that you actually have the disease is only 95/5090, or about 2%, even with a pretty good test. The reason is that the sheer number of people without the disease is so much greater that even a low % of false positives translates into large absolute numbers.
For rare diseases, tests are often much better at ruling out disease than ruling it in. If you had tested negative, your probability of having the disease would be 5/94,910, or 0.0001%.
Hope that’s right – would be a shame to get it wrong in front of such an audience : ). And of course, if you mean something else by “correctly detects the disease”, then you’d have to adjust the numbers accordingly.
cheers,
Miranda
From: Evidence based health (EBH) [mailto:[log in to unmask]] On Behalf Of Paul Glasziou
Sent: Thursday, 10 February 2011 10:55 AM
To: [log in to unmask]
Subject: Re: Question about screening and denominators
Hi Simon,
Brian is right that you need to know more to answer the question.
If the "test which correctly detects the disease 95%of the time" and we interpret that as the sensitivity.
If the specificity is 5% then the test is useless - a Youden index (sensitivity + specicity - 100%)
and so after a positive test the chance of disease is still 1:1000
Paul Glasziou
PS If instead the 95% is the overall accuracy, then just call everyone negative and the test is 99.9% accurate ;-)
On 2/10/2011 9:31 AM, Brian Alper MD wrote:
Here’s my quick thoughts without doing any math or providing formulas.
More than one answer is possible depending on how one interprets “correctly detects the disease 95% of the time”
If you know the sensitivity (% correct in patients with known disease), specificity (% correct in patients known to not have disease) and disease prevalence (1:1000 in this example) you can answer the question.
But overall measures of diagnostic accuracy (% correct overall based on combining sensitivity and specificity) could reach 95% through different combinations of sensitivity and specificity. The different combinations could lead to different results for your question (a measure of positive predictive value)
Brian S. Alper, MD, MSPH
Editor-in-Chief, DynaMed (www.ebscohost.com/dynamed)
From: Evidence based health (EBH) [mailto:[log in to unmask]] On Behalf Of Simon Hatcher
Sent: Wednesday, February 09, 2011 6:22 PM
To: [log in to unmask]
Subject: Question about screening and denominators
I had this discussion with a friend over a beer last night and we couldn't agree on the answer. Here's the scenario:
The incidence of a disease in a population is 1:1000
There is a test which correctly detects the disease 95%of the time
If I test positive with the test what is my risk of having the disease?
Be interested in any thoughts on the "correct" answer.
Cheers
Simon
-- Paul GlasziouBond UniversityQld, Australia 4229