Maybe I should have just used the negative predictive value in the first place (as always ain't hindsight great).

However I think that the probablities and likelihood ratios will be a potent force in the future of emergency medicine - our trainees must be good at thinking in this way (it is for clinicians not statisticians). There are two reasons why using the simple PPV and NPV is not good enough. 

1) PPV and NPV (like sensitivity and specificity) are dependent on the prevelance of the disease in the population - as elegantly illustrated by Michael. (can I therefore transfer your NPV and PPV onto my population?). The likelihood ratios are independent of the population prevelance of the disease therefore can be directly applied to different populations.

2) The PPV and NPV (and sens and spec) tell us about a population average NOT about your individual patient. The patient does not want to know about the population's risk - he or she wants to know about their own risk. I think that on clincial grounds I know for most patients whether they are at more or less risk than the normal population and alter my advice accordingly. I bet you do the same for just about every patient that you see - unconsciously you are using pre and post test probabilities. All the statistics does is to put some rough numbers onto this process. So probabilities and likelihood ratios allow you to get one step nearer to tailoring your knowledge about the power of a test to the individual patient that you are investigating.

Tim.

PS. OK. You are right. I was getting carried away. The sensitivity and specificity  are not completely useless and DO tell you something about a test - but only if you can have the mental agility needed to work to the NPV and PPV as you suggest. Why use sensitivity and specificity when NPV and PPV are so much more in tune with what we are doing in emergency medicine?


-----Original Message-----
From: Accident and Emergency Academic List
[mailto:[log in to unmask]]On Behalf Of Adrian Fogarty
Sent: 02 March 2006 02:36
To: [log in to unmask]
Subject: Re: SAH


Thanks for all that Tim, very helpful. Just to take one of your points: 
"Negative predictive value is 99.9% (i.e. if the CT is negative only 0.1% of 
patients actually has an SAH)"

I'm thinking to myself, why didn't you just say this in the first place? 
(i.e. instead of all this confusing stuff about likelihood ratios, pre- and 
post-test probabilities!)  I mean, if the negative predictive value tells us 
that only 1 in 1000 patients with a clear scan have SAH, isn't that the only 
stats we really need to know, i.e. that's the bottom line when it comes to 
this sort of clinical conundrum? Or does the negative predictive value vary 
with the "population prevalence"? (I'm assuming it might but this is where 
my stats gets woolly!)

My rudimentary stats tells me that PPV is closely related to specificity, 
and NPV is closely related to sensitivity, so is it really correct for you 
to say sens and spec are useless at interpreting results of a test? In other 
words, a highly sensitive test (with few false negatives) will also produce 
a high NPV, while a highly specific test (with few false positives) will 
also produce a high PPV, won't it? Am I missing something here?

Apologies to the 49% of list lurkers who already know all this stuff, and 
the other 49% who don't know but don't care, leaving a handful of us folks 
who kind of know a little bit but want to know more! And sorry about your 
headache Tim...

AF

----- Original Message ----- 
From: "Coats Tim - Professor of Emergency Medicine" 
<[log in to unmask]>
To: <[log in to unmask]>
Sent: Wednesday, March 01, 2006 1:57 PM
Subject: Re: SAH


Adrian,

I think that you have confused the meaning of sensitivity. A 98% sensitivity 
certainly does not mean that 1 in 50 of patients with a negative scan will 
none the less have an SAH.
I think that you have also confused the meaning of a likelihood ratio. A 
ratio of 0.02 certainly does not mean that 1 in 50 of patients with a 
negative scan will none the less have an SAH.

A 98% sensitivity means the 98% of patients WITH SAH will have a positive 
scan. This is a completely useless figure in patient decision making. We 
don't want to know the proportion of patients WITH THE DISEASE that have a 
positive test. As clinicians we want to know the proportion of patients WITH 
A NEGATIVE scan that hve the disease.

Sensitivity and specificity are useless in interpreting the results of a 
test.

We need to use the negative predictive value (my patient has a negative 
test, in what proportion does this really mean that they don't have the 
disease) and the positive predictive value (my patient has a positive test, 
in what proportion does this mean that they have the disease).

For CT scanning (assuming the population prevelance of 5%):
Positive predictive value is 99.8% (ie if CT is positive I can say that 
99.8% of patients actually have SAH)
Negative predictive value is 99.9% (ie. if the CT is negative only 0.1% of 
patients actually has an SAH)
Sensitivity is 98%
Specificity is nearly 100%

If I want to tailor this to an individual patient (who may have a risk that 
I judge is more or less than the population average of 5%) I can use the 
likelihood ratio to work out a more individual calculation.

BOTTOM LINE: If the CT scan is negative 0.1% of patients will have an SAH 
(ie 1 in 1000).

SECOND BOTTOM LINE: Sensitivity and specificity are very misleading ways of 
describing a diagnostic test. I have no idea why they are so often quoted.

I feel a headache coming on, wait a minute, whats the probability......

Tim.


-----Original Message-----
From: Accident and Emergency Academic List
[mailto:[log in to unmask]]On Behalf Of Adrian Fogarty
Sent: 01 March 2006 01:20
To: [log in to unmask]
Subject: Re: SAH


No, the other way round; I was talking patients while you were talking
populations, I think! (see below)

My definition of likelihood ratio: well, I was "paraphrasing" really - and
omitted the denominator!  I understand negative likelihood ratio to be "the
probability of an individual with the condition having a negative test -
over - the probability of an individual without the condition having a
negative test".  Now, since the latter (the denominator) is very close to
unity, I have taken the liberty of omitting it for clarity's sake. Hence,
when I'm told the negative likelihood ratio of CT for SAH is 0.02, that
tells me that 1 in 50 of patients with a negative scan will nonetheless have
an SAH.

Yes, I gather you can also express this as (1-sens)/spec as you suggest, but
it can be put into words; I work better with words you might have noticed!
Again, here we have a highly sensitive test (in the order of 95-98%
depending on the scanner etc) but an extremely specific test (you don't
often see a bleed that's not there!). So again, you can virtually omit the
denominator and you're left with (1-sens) which comes back to the 0.02
figure (1-0.98) or 1 in 50.

I think our difficulty - as you've alluded - is that you're "applying" this
to a 5% pre-test probability from your historical population. But I have
some difficulty with this, as you only know they're 5% after their scan.
Surely before you scan an individual you must "subconsciously" think they're
much higher risk than 5%? And surely the patient thinks they're "near 100%"
until proven otherwise, and so does their physician who's ordering the scan?
They're thinking to themselves: "if I've got SAH, what are the chances this
test will miss it?". And the answer's 0.02 x 100 = 2 which is 1 in 50.
Hence, why would they then refuse an LP based on a 1-in-50 chance they've
still got SAH?

But I do see your broader point - and herein lies the crux of the matter -
which is to take into account all the true negatives mixed in there with
that one true SAH that your scan's missed. So fine, if your population only
has a yield of, say, 50 positive scans per 1000 patients, then you're going
to have 950 patients left over anxious for a diagnosis, yet we know only 1
of those is a true positive (1 in 50). That means we need 950 LPs to find
that one positive SAH. And that's the very compelling population argument! I
suppose it all comes down to the remarkably low yield of 5% in your
population which rather surprised me. In other words, if we were working in
a population where, say, 50% of our scans were positive, then you'd have to
LP the rest, as 1 in 50 of them would have SAH.

So there goes my rather simplistic view of stats...it gets me into all sorts
of trouble you know.

AF



----- Original Message ----- 
From: "Coats Tim - Professor of Emergency Medicine"
<[log in to unmask]>
To: <[log in to unmask]>
Sent: Tuesday, February 28, 2006 6:50 PM
Subject: Re: SAH


Adrian,
We are not seeing eye to eye on this as I am thinking about the
probabilities for an indivdual patient and I wonder if you are thinking
about probabilities in population terms.

The negative likelihood ratio of 0.02 is a property of the test and is the
same for all individuals (and is independent of the prevelance of the
disease in the population).

The pre-test probability is what you use clinical skill to attach to an
individual patient. An individual patient will not have the 'population
average' pre-test probability - you will probably find more or less worrying
features in their history. This could be from nearly 0% (here is some
paracetamol bye bye) to nearly 100%(you have a SAH).

The population prevelance (the 0.05 Sally Clarke figure) is not relevant
here (in calculating the likelihood ratio the prevelance in the population
is present on both sides of the equation and therefore cancels out).

I don't really agree with your definition of Likelihood ratio as "the
probability of an individual with the condition having a negative test".
LR-ve = (1-sens)/spec. I am not sure that I can put this into words - it is
a mathematical number which cannot really be equated to the sort of terms
that you are using.

Tim.



-----Original Message-----
From: Accident and Emergency Academic List
[mailto:[log in to unmask]]On Behalf Of Adrian Fogarty
Sent: 28 February 2006 10:38
To: [log in to unmask]
Subject: Re: SAH


But I still think you're overegging it, doing a "Sally Clark" as it were.
First of all, your whole premise of the pretest probability of 5% is based
on CT results. Nevertheless, let's accept for a moment that of a LASH
population (see, you've got me doing it now!), 5% will end up having the
target disease, so 5% is your pretest probability. But then you're stating
that the negative likelihood ratio for CT is 0.02. The very definition of
negative likelihood ratio is tantamount to "the probability of an individual
with the condition having a negative test" so there's no need to factor the
0.02 by 0.05, if the figure of 0.02 has already been established as it were.

Yes, by all means factor in the negative likelihood ratio of LP and factor
that by 0.02, but that will give you those patients who have SAH but are
negative for both CT and LP. And yes they will indeed be rare, but that's
not who we're interested in here; we're interested in the vastly greater
number who will be LP positive after CT negative.

Maybe I'm wrong but if Tim's saying that only 1 in a 1000 negative CTs end
up having SAH then maybe PB's right, we shouldn't be doing LPs at all. But
from my understanding, the figures are much higher than 1 in 1000. Am I
suffering from mental entrenchment syndrome?

AF

----- Original Message ----- 
From: "Andrew Webster" <[log in to unmask]>
To: <[log in to unmask]>
Sent: Tuesday, February 28, 2006 8:02 AM
Subject: Re: SAH


> >
>> And Tim, you're a better statistician (and neurosurgeon) than me, I've no
>> doubt, but I don't quite follow your maths approach here. If the negative
>> likelihood ratio of CT for SAH is 0.02, that basically means that of all
>> the
>> negative CTs we get (for LASH!), 1 in 50 of those patients will still
>> have
>> an SAH, doesn't it? (correct me if I'm wrong here...) It seems to me that
>> you're citing a pretest probability of 5% to the scan-clear patients
>> (which
>> is fair enough), but you're then factoring in the negative likelihood
>> ratio
>> of CT testing itself, giving an incredible 1-in-1000 result. That's not
>> rational to my mind. You're citing a pretest probability based on the
>> results of the same test, a circular argument if you like. Surely it
>> would
>> be more appropriate to take that pre-test probability and then factor in
>> the
>> positive likelihood ratio of LP wouldn't it? What I'm saying is, for
>> every
>> 50 patients you get coming out of the CT with a clear scan, you're going
>> to
>> have one who actually has SAH and that one is very likely to be picked up
>> by
>> the subsequent LP (assuming LP is fairly sensitive when done at the
>> correct
>> time etc). So it can't be 1 in a 1000, it's got to be closer to 1 in 50,
>> doesn't it? (which might explain why PB's had two of them in the last few
>> years...)
> No Tim is saying before we do any tests ourselves the pretest probability
> of the patient having a SAH is 5%. With a negative CT the likelihood ratio
> of 0.02 the post test probability is 0.1% or 1 in a 1000. Which is why you
> need to do 1 in 1000 lp's to find a positive LP
>
> Andy Webster

This e-mail,  including any attached files, may contain confidential and /
or privileged information and is intended for the exclusive use of the
addressee(s) printed above. If you are not the addressee(s), any
unauthorised review, disclosure, reproduction, other dissemination or use of
this e-mail, or taking of any action in reliance upon the information
contained herein, is strictly prohibited. If this e-mail has been sent to
you in error, please return to the sender. No guarantee can be given that
the contents of this email are virus free - The University Hospitals of
Leicester NHS Trust cannot be held responsible for any failure by the
recipient(s) to test for viruses before opening any attachments.   The
information contained in this e-mail may be the subject of public disclosure
under the Freedom of Information Act 2000 -  unless legally exempt from
disclosure, the confidentiality of this e-mail and your reply cannot be
guaranteed. Copyright in this email and any attachments created by us
remains vested in the University Hospitals of Leicester NHS Trust.

This e-mail,  including any attached files, may contain confidential and / 
or privileged information and is intended for the exclusive use of the 
addressee(s) printed above. If you are not the addressee(s), any 
unauthorised review, disclosure, reproduction, other dissemination or use of 
this e-mail, or taking of any action in reliance upon the information 
contained herein, is strictly prohibited. If this e-mail has been sent to 
you in error, please return to the sender. No guarantee can be given that 
the contents of this email are virus free - The University Hospitals of 
Leicester NHS Trust cannot be held responsible for any failure by the 
recipient(s) to test for viruses before opening any attachments.   The 
information contained in this e-mail may be the subject of public disclosure 
under the Freedom of Information Act 2000 -  unless legally exempt from 
disclosure, the confidentiality of this e-mail and your reply cannot be 
guaranteed. Copyright in this email and any attachments created by us 
remains vested in the University Hospitals of Leicester NHS Trust. 

This e-mail,  including any attached files, may contain confidential and / or privileged information and is intended for the exclusive use of the addressee(s) printed above. If you are not the addressee(s), any unauthorised review, disclosure, reproduction, other dissemination or use of this e-mail, or taking of any action in reliance upon the information contained herein, is strictly prohibited. If this e-mail has been sent to you in error, please return to the sender. No guarantee can be given that the contents of this email are virus free - The University Hospitals of Leicester NHS Trust cannot be held responsible for any failure by the recipient(s) to test for viruses before opening any attachments.   The information contained in this e-mail may be the subject of public disclosure under the Freedom of Information Act 2000 -  unless legally exempt from disclosure, the confidentiality of this e-mail and your reply cannot be guaranteed. Copyright in this email and any attachments created by us remains vested in the University Hospitals of Leicester NHS Trust.