 |
 |
Maybe I should have just used the negative predictive value in the first place (as always ain't hindsight great). However I think that the probablities and likelihood ratios will be a potent force in the future of emergency medicine - our trainees must be good at thinking in this way (it is for clinicians not statisticians). There are two reasons why using the simple PPV and NPV is not good enough. 1) PPV and NPV (like sensitivity and specificity) are dependent on the prevelance of the disease in the population - as elegantly illustrated by Michael. (can I therefore transfer your NPV and PPV onto my population?). The likelihood ratios are independent of the population prevelance of the disease therefore can be directly applied to different populations. 2) The PPV and NPV (and sens and spec) tell us about a population average NOT about your individual patient. The patient does not want to know about the population's risk - he or she wants to know about their own risk. I think that on clincial grounds I know for most patients whether they are at more or less risk than the normal population and alter my advice accordingly. I bet you do the same for just about every patient that you see - unconsciously you are using pre and post test probabilities. All the statistics does is to put some rough numbers onto this process. So probabilities and likelihood ratios allow you to get one step nearer to tailoring your knowledge about the power of a test to the individual patient that you are investigating. Tim. PS. OK. You are right. I was getting carried away. The sensitivity and specificity are not completely useless and DO tell you something about a test - but only if you can have the mental agility needed to work to the NPV and PPV as you suggest. Why use sensitivity and specificity when NPV and PPV are so much more in tune with what we are doing in emergency medicine? -----Original Message----- From: Accident and Emergency Academic List [mailto:[log in to unmask]]On Behalf Of Adrian Fogarty Sent: 02 March 2006 02:36 To: [log in to unmask] Subject: Re: SAH Thanks for all that Tim, very helpful. Just to take one of your points: "Negative predictive value is 99.9% (i.e. if the CT is negative only 0.1% of patients actually has an SAH)" I'm thinking to myself, why didn't you just say this in the first place? (i.e. instead of all this confusing stuff about likelihood ratios, pre- and post-test probabilities!) I mean, if the negative predictive value tells us that only 1 in 1000 patients with a clear scan have SAH, isn't that the only stats we really need to know, i.e. that's the bottom line when it comes to this sort of clinical conundrum? Or does the negative predictive value vary with the "population prevalence"? (I'm assuming it might but this is where my stats gets woolly!) My rudimentary stats tells me that PPV is closely related to specificity, and NPV is closely related to sensitivity, so is it really correct for you to say sens and spec are useless at interpreting results of a test? In other words, a highly sensitive test (with few false negatives) will also produce a high NPV, while a highly specific test (with few false positives) will also produce a high PPV, won't it? Am I missing something here? Apologies to the 49% of list lurkers who already know all this stuff, and the other 49% who don't know but don't care, leaving a handful of us folks who kind of know a little bit but want to know more! And sorry about your headache Tim... AF ----- Original Message ----- From: "Coats Tim - Professor of Emergency Medicine" <[log in to unmask]> To: <[log in to unmask]> Sent: Wednesday, March 01, 2006 1:57 PM Subject: Re: SAH Adrian, I think that you have confused the meaning of sensitivity. A 98% sensitivity certainly does not mean that 1 in 50 of patients with a negative scan will none the less have an SAH. I think that you have also confused the meaning of a likelihood ratio. A ratio of 0.02 certainly does not mean that 1 in 50 of patients with a negative scan will none the less have an SAH. A 98% sensitivity means the 98% of patients WITH SAH will have a positive scan. This is a completely useless figure in patient decision making. We don't want to know the proportion of patients WITH THE DISEASE that have a positive test. As clinicians we want to know the proportion of patients WITH A NEGATIVE scan that hve the disease. Sensitivity and specificity are useless in interpreting the results of a test. We need to use the negative predictive value (my patient has a negative test, in what proportion does this really mean that they don't have the disease) and the positive predictive value (my patient has a positive test, in what proportion does this mean that they have the disease). For CT scanning (assuming the population prevelance of 5%): Positive predictive value is 99.8% (ie if CT is positive I can say that 99.8% of patients actually have SAH) Negative predictive value is 99.9% (ie. if the CT is negative only 0.1% of patients actually has an SAH) Sensitivity is 98% Specificity is nearly 100% If I want to tailor this to an individual patient (who may have a risk that I judge is more or less than the population average of 5%) I can use the likelihood ratio to work out a more individual calculation. BOTTOM LINE: If the CT scan is negative 0.1% of patients will have an SAH (ie 1 in 1000). SECOND BOTTOM LINE: Sensitivity and specificity are very misleading ways of describing a diagnostic test. I have no idea why they are so often quoted. I feel a headache coming on, wait a minute, whats the probability...... Tim. -----Original Message----- From: Accident and Emergency Academic List [mailto:[log in to unmask]]On Behalf Of Adrian Fogarty Sent: 01 March 2006 01:20 To: [log in to unmask] Subject: Re: SAH No, the other way round; I was talking patients while you were talking populations, I think! (see below) My definition of likelihood ratio: well, I was "paraphrasing" really - and omitted the denominator! I understand negative likelihood ratio to be "the probability of an individual with the condition having a negative test - over - the probability of an individual without the condition having a negative test". Now, since the latter (the denominator) is very close to unity, I have taken the liberty of omitting it for clarity's sake. Hence, when I'm told the negative likelihood ratio of CT for SAH is 0.02, that tells me that 1 in 50 of patients with a negative scan will nonetheless have an SAH. Yes, I gather you can also express this as (1-sens)/spec as you suggest, but it can be put into words; I work better with words you might have noticed! Again, here we have a highly sensitive test (in the order of 95-98% depending on the scanner etc) but an extremely specific test (you don't often see a bleed that's not there!). So again, you can virtually omit the denominator and you're left with (1-sens) which comes back to the 0.02 figure (1-0.98) or 1 in 50. I think our difficulty - as you've alluded - is that you're "applying" this to a 5% pre-test probability from your historical population. But I have some difficulty with this, as you only know they're 5% after their scan. Surely before you scan an individual you must "subconsciously" think they're much higher risk than 5%? And surely the patient thinks they're "near 100%" until proven otherwise, and so does their physician who's ordering the scan? They're thinking to themselves: "if I've got SAH, what are the chances this test will miss it?". And the answer's 0.02 x 100 = 2 which is 1 in 50. Hence, why would they then refuse an LP based on a 1-in-50 chance they've still got SAH? But I do see your broader point - and herein lies the crux of the matter - which is to take into account all the true negatives mixed in there with that one true SAH that your scan's missed. So fine, if your population only has a yield of, say, 50 positive scans per 1000 patients, then you're going to have 950 patients left over anxious for a diagnosis, yet we know only 1 of those is a true positive (1 in 50). That means we need 950 LPs to find that one positive SAH. And that's the very compelling population argument! I suppose it all comes down to the remarkably low yield of 5% in your population which rather surprised me. In other words, if we were working in a population where, say, 50% of our scans were positive, then you'd have to LP the rest, as 1 in 50 of them would have SAH. So there goes my rather simplistic view of stats...it gets me into all sorts of trouble you know. AF ----- Original Message ----- From: "Coats Tim - Professor of Emergency Medicine" <[log in to unmask]> To: <[log in to unmask]> Sent: Tuesday, February 28, 2006 6:50 PM Subject: Re: SAH Adrian, We are not seeing eye to eye on this as I am thinking about the probabilities for an indivdual patient and I wonder if you are thinking about probabilities in population terms. The negative likelihood ratio of 0.02 is a property of the test and is the same for all individuals (and is independent of the prevelance of the disease in the population). The pre-test probability is what you use clinical skill to attach to an individual patient. An individual patient will not have the 'population average' pre-test probability - you will probably find more or less worrying features in their history. This could be from nearly 0% (here is some paracetamol bye bye) to nearly 100%(you have a SAH). The population prevelance (the 0.05 Sally Clarke figure) is not relevant here (in calculating the likelihood ratio the prevelance in the population is present on both sides of the equation and therefore cancels out). I don't really agree with your definition of Likelihood ratio as "the probability of an individual with the condition having a negative test". LR-ve = (1-sens)/spec. I am not sure that I can put this into words - it is a mathematical number which cannot really be equated to the sort of terms that you are using. Tim. -----Original Message----- From: Accident and Emergency Academic List [mailto:[log in to unmask]]On Behalf Of Adrian Fogarty Sent: 28 February 2006 10:38 To: [log in to unmask] Subject: Re: SAH But I still think you're overegging it, doing a "Sally Clark" as it were. First of all, your whole premise of the pretest probability of 5% is based on CT results. Nevertheless, let's accept for a moment that of a LASH population (see, you've got me doing it now!), 5% will end up having the target disease, so 5% is your pretest probability. But then you're stating that the negative likelihood ratio for CT is 0.02. The very definition of negative likelihood ratio is tantamount to "the probability of an individual with the condition having a negative test" so there's no need to factor the 0.02 by 0.05, if the figure of 0.02 has already been established as it were. Yes, by all means factor in the negative likelihood ratio of LP and factor that by 0.02, but that will give you those patients who have SAH but are negative for both CT and LP. And yes they will indeed be rare, but that's not who we're interested in here; we're interested in the vastly greater number who will be LP positive after CT negative. Maybe I'm wrong but if Tim's saying that only 1 in a 1000 negative CTs end up having SAH then maybe PB's right, we shouldn't be doing LPs at all. But from my understanding, the figures are much higher than 1 in 1000. Am I suffering from mental entrenchment syndrome? AF ----- Original Message ----- From: "Andrew Webster" <[log in to unmask]> To: <[log in to unmask]> Sent: Tuesday, February 28, 2006 8:02 AM Subject: Re: SAH > > >> And Tim, you're a better statistician (and neurosurgeon) than me, I've no >> doubt, but I don't quite follow your maths approach here. If the negative >> likelihood ratio of CT for SAH is 0.02, that basically means that of all >> the >> negative CTs we get (for LASH!), 1 in 50 of those patients will still >> have >> an SAH, doesn't it? (correct me if I'm wrong here...) It seems to me that >> you're citing a pretest probability of 5% to the scan-clear patients >> (which >> is fair enough), but you're then factoring in the negative likelihood >> ratio >> of CT testing itself, giving an incredible 1-in-1000 result. That's not >> rational to my mind. You're citing a pretest probability based on the >> results of the same test, a circular argument if you like. Surely it >> would >> be more appropriate to take that pre-test probability and then factor in >> the >> positive likelihood ratio of LP wouldn't it? What I'm saying is, for >> every >> 50 patients you get coming out of the CT with a clear scan, you're going >> to >> have one who actually has SAH and that one is very likely to be picked up >> by >> the subsequent LP (assuming LP is fairly sensitive when done at the >> correct >> time etc). So it can't be 1 in a 1000, it's got to be closer to 1 in 50, >> doesn't it? (which might explain why PB's had two of them in the last few >> years...) > No Tim is saying before we do any tests ourselves the pretest probability > of the patient having a SAH is 5%. With a negative CT the likelihood ratio > of 0.02 the post test probability is 0.1% or 1 in a 1000. Which is why you > need to do 1 in 1000 lp's to find a positive LP > > Andy Webster This e-mail, including any attached files, may contain confidential and / or privileged information and is intended for the exclusive use of the addressee(s) printed above. If you are not the addressee(s), any unauthorised review, disclosure, reproduction, other dissemination or use of this e-mail, or taking of any action in reliance upon the information contained herein, is strictly prohibited. If this e-mail has been sent to you in error, please return to the sender. No guarantee can be given that the contents of this email are virus free - The University Hospitals of Leicester NHS Trust cannot be held responsible for any failure by the recipient(s) to test for viruses before opening any attachments. The information contained in this e-mail may be the subject of public disclosure under the Freedom of Information Act 2000 - unless legally exempt from disclosure, the confidentiality of this e-mail and your reply cannot be guaranteed. Copyright in this email and any attachments created by us remains vested in the University Hospitals of Leicester NHS Trust. This e-mail, including any attached files, may contain confidential and / or privileged information and is intended for the exclusive use of the addressee(s) printed above. If you are not the addressee(s), any unauthorised review, disclosure, reproduction, other dissemination or use of this e-mail, or taking of any action in reliance upon the information contained herein, is strictly prohibited. If this e-mail has been sent to you in error, please return to the sender. No guarantee can be given that the contents of this email are virus free - The University Hospitals of Leicester NHS Trust cannot be held responsible for any failure by the recipient(s) to test for viruses before opening any attachments. The information contained in this e-mail may be the subject of public disclosure under the Freedom of Information Act 2000 - unless legally exempt from disclosure, the confidentiality of this e-mail and your reply cannot be guaranteed. Copyright in this email and any attachments created by us remains vested in the University Hospitals of Leicester NHS Trust. This e-mail, including any attached files, may contain confidential and / or privileged information and is intended for the exclusive use of the addressee(s) printed above. If you are not the addressee(s), any unauthorised review, disclosure, reproduction, other dissemination or use of this e-mail, or taking of any action in reliance upon the information contained herein, is strictly prohibited. If this e-mail has been sent to you in error, please return to the sender. No guarantee can be given that the contents of this email are virus free - The University Hospitals of Leicester NHS Trust cannot be held responsible for any failure by the recipient(s) to test for viruses before opening any attachments. The information contained in this e-mail may be the subject of public disclosure under the Freedom of Information Act 2000 - unless legally exempt from disclosure, the confidentiality of this e-mail and your reply cannot be guaranteed. Copyright in this email and any attachments created by us remains vested in the University Hospitals of Leicester NHS Trust.