 |
 |
The issue of conditional independence of diagnostic tests is a separate issue as to whether one should report Bayesian posterior distributions as evidence. However, since it has been raised, and in illustration of Hanan Bell's point, let me give a simple example where conditional independence is not true. Two colleagues give an opinion on the presence or absence of a disease. One is a junior doctor, with little training, the other is a very experienced senior colleague trained in all the arts of EBM. They both give a positive diagnosis. Let us call the first 'evidence' E1 and the second 'evidence' E2. Presumably the posterior probability of disease is increased. But it might be the case that given E2, E1 is irrelevant whereas given E1, E2 still has some relevance. In other words E1 would increase your posterior belief of disease presence and E2 even more so but once you had E2, E1 would have no further effect. It is important to understand here that the joint effect of E1 and E2 does not depend on the order in which they are obtained. It is simply that the joint likelihood ratio cannot be obtained by simple mucltiplication of the marginal ratios. As one who is neither American nor British, I would like to point out that this issue does not depend on whether you live in a republic or a constitutional monarchy but I am all in favour of discussing it in the bar! Regards Stephen ________________________________________ From: Evidence based health (EBH) [[log in to unmask]] On Behalf Of Dan Sontheimer, MD, MBA [[log in to unmask]] Sent: 09 November 2010 04:17 To: [log in to unmask] Subject: Re: Bayesians, likelihood and evidence Pardon the Americanism... So three Bayesians walk into a bar and each try to remember their order of entry after waking up on the lawn of Buckingham palace. Back to reality Dan Sontheimer, MD, MBA CMO, CoxHealth On Nov 8, 2010, at 7:18 PM, "Dr. Carlos Cuello" <[log in to unmask]<mailto:[log in to unmask]>> wrote: Thanks, Ben and all for your responses I am citing Hanan Bell (sorry, Hanan ;) ) I think it might help us all understand; I have always had this question of consequential diagnostic tests and LR. Pauker and Kasirer never mentioned this on their classic article. When I teach diagnostic tests I only use one test, and one LR (one move). But this does not happen in real life. So, could we ever use two or more consequential LRs (moves) without all the complexity? "It is more complex than you thought. The answer depends primarily on how you got the likelihood for the second test. If that likelihood was computed based on patients who had already been negative on the first test, you are fine. If the sensitivity and specificity of the second test are the same regardless of whether a patient is positive or negative on the first test, you are also fine. This is the situation where the tests are truly independent. However, if the tests are correlated at all, then the approach you suggest is not appropriate and you need to take that correlation into account. You wouldn’t go back to the original prior, but you would have to be careful about the LR you use for the second test. It wouldn’t be the same as one computed on the original population. You’d need additional information about the interrelation of the two tests. Hanan Bell Technical Advisor, Diagnostics Assessment Programme NICE" On Mon, Nov 8, 2010 at 18:09, Djulbegovic, Benjamin <<mailto:[log in to unmask]>[log in to unmask]<mailto:[log in to unmask]>> wrote: Hi Carlos, I am jumping in again because I think this debate is essential for EBMers (particularly EBM teachers). The way I understood Stephen's explanation (I am yet to read the papers he recommended- so, I still may be collosally wrong here), if your colleague told you: "I have performed a diagnostic test, and my post-test probability of appendicitis is 4%", then you will not be able to use this information because a) you are not told which test he/she performed, b) what was his/her priors. However, in your example, since you are the only one who is making calculations (i.e.. the person who knows his priors and evidence on test1 and test2), you should be able to use posteriors after the first test as the priors for the second one. Stephen, please comment, if this is correct interpretation of your explanations. thanks ben Dan Sontheimer, MD, MBA Chief Medical Officer, Executive Administration CoxHealth [log in to unmask] Phone: 69-7179 Mobile: 459-3512 Fax: (417) 269-3104 __________ ________________________________________ From: Evidence based health (EBH) [<mailto:[log in to unmask]>[log in to unmask]<mailto:[log in to unmask]>] On Behalf Of Stephen Senn [<mailto:[log in to unmask]>[log in to unmask]<mailto:[log in to unmask]>] Sent: Monday, November 08, 2010 6:08 PM To: <mailto:[log in to unmask]> [log in to unmask]<mailto:[log in to unmask]> Subject: Re: Bayesians, likelihood and evidence Dear Carlos, Your example raises issues I don't want to go into regarding conditional independence but quite apart from anything else there is a difference between a) using results that have been reported by others to make my Bayesian decision b) reporting my evidence in a way that will help others come to their Bayesian decision. I don't think that your example is in the second camp (you are making the decision for the patient) and in fact at least two of the statistics you quoted are not prior nor even posterior probabilities; they are non-Bayesian (negLR = 0.06) and (posLR=40). But maybe I have imsunderstood So the issue is this, just because you want to be SBM for a) does not mean you should not be EBM for b) Regards Stephen ________________________________________ From: Evidence based health (EBH) [<mailto:[log in to unmask]>[log in to unmask]<mailto:[log in to unmask]>] On Behalf Of Dr. Carlos Cuello [<mailto:[log in to unmask]>[log in to unmask]<mailto:[log in to unmask]>] Sent: 08 November 2010 22:19 To: <mailto:[log in to unmask]> [log in to unmask]<mailto:[log in to unmask]> Subject: Re: Bayesians, likelihood and evidence Apologies if I sound naive but... in mortal terms, translating this to the daily clinical practice... If I have a pediatric patient with abdominal pain and a 40% chance of having appendicitis, (I obtained this prior probability from my "local prevalence in a previous study" and before performing any tests). I apply the Kharbanda rule for appendicitis, which is negative (negLR = 0.06), that means I have a post-test probability of 4%. I THEN apply an ultrasound test (yes, even with that low probability of appendicitis), and suppose is positive (posLR=40). Which prevalence or pre-test probability I must use? I always teach that you perform your new test (if you decide to perform it) with this new probability of 4%. Which, in this case the positive ultrasound gives a 60% chance of having appendicitis. But if you take the orignal 40% you obtain a 97% chance of having appendicitis. Or is it more complex than previously thought and I have to buy lots of books and see a lot of bayesian strategies that it would never be used by the busy clinician? On Mon, Nov 8, 2010 at 13:59, Stephen Senn <<mailto:[log in to unmask]>[log in to unmask]<mailto:[log in to unmask]><mailto:<mailto:[log in to unmask]>[log in to unmask]<mailto:[log in to unmask]>>> wrote: Dear Ben, It basically boils down to this. Evidence cannot be used twice, even prior evidence. So in the following suppose that we have a Bayersian i who collects evidence at E1,E2 etc staring with a prior P0_i . It certainly is true that she can add the evidence (schematically) to her prior as follows P1_i =P0_i+E1, P2_i=P1_i+E2, or equivalently P2_i=P0_i+E1+E2. Thus for, example it does not matter for the Bayesian whether trials are added one by one or summarised in a meta-analysis and then added in one go. Now, however, suppose it is the case that this Bayesian, after having obtained evidence E1 and thus having a distribution of belief P1_i meets another Bayesian, Bayesian j, who has just summarised some evidence E2. Unfortunately, however, this Bayesian does not report E2 but reports his Bayesian posterior P2_j. This posterior distribution cannot be used by the first Bayesian because she needs to know a) does it include E1? b) does it include P0_i (or any element of this)? If so, these have to be subtracted from P2_j. However, subtracting the requisite elements leaves us just with E2, which is not a Bayesian statement. You may be interested to look at Guernsey McPearson's explanation here <http://www.senns.demon.co.uk/wprose.html#Priors> http://www.senns.demon.co.uk/wprose.html#Priors . He does seem to have it in rather for Bayesians and just to even things up you may wish to see what he has to say about the Archie Association <http://www.senns.demon.co.uk/wprose.html#Archies> http://www.senns.demon.co.uk/wprose.html#Archies (I should think that there will plenty in these to offend both SBM and EBM.) Regards Stephen ________________________________________ From: Djulbegovic, Benjamin [<mailto:[log in to unmask]>[log in to unmask]<mailto:[log in to unmask]><mailto:<mailto:[log in to unmask]>[log in to unmask]<mailto:[log in to unmask]>>] Sent: 08 November 2010 19:23 To: Stephen Senn; <mailto:[log in to unmask]> [log in to unmask]<mailto:[log in to unmask]><mailto:<mailto:[log in to unmask]>[log in to unmask]<mailto:[log in to unmask]>> Subject: RE: SBM-EBM shouting match Hi Stephen, I found what you wrote very interesting (that "Bayesians cannot use a set of posterior distributions from various studies as the raw input to a Bayesian analysis.") I have seen many, many papers that claim that "these (posteriors) should be used as priors"...and we have certainly been teaching in clinical medicine for years how posterior results of one diagnostic test should/can be used as the prior for the second diagnostic test (at least, in serially applied tests...). Would you mind explaining or sending a reference where this was discussed so that I can correct my naïve Bayesian views? Thanks ben -----Original Message----- From: Evidence based health (EBH) [mailto:<mailto:[log in to unmask]>[log in to unmask]<mailto:[log in to unmask]><mailto:<mailto:[log in to unmask]>[log in to unmask]<mailto:[log in to unmask]>>] On Behalf Of Stephen Senn Sent: Monday, November 08, 2010 6:30 AM To: <mailto:[log in to unmask]> [log in to unmask]<mailto:[log in to unmask]><mailto:<mailto:[log in to unmask]>[log in to unmask]<mailto:[log in to unmask]>> Subject: Re: SBM-EBM shouting match Richard Royall is a statistician who has had much to say about evidence. He points out that on having completed a study there are three rather different questions we might wish to answer. 1. What do I now believe? 2. What should I now do? 3. What is the evidence of this study? Bayesians are concerned with 1 and (to the extent that they consider utilities also) with 2. These are good and laudable concerns. However, it as an irony of Bayesian analysis that naive Bayesians are wont to overlook that as the late George Barnard, the father of the likelihood principle, pointed out, Bayesians cannot use a set of posterior distributions from various studies as the raw input to a Bayesian analysis. They can often make better use of standard frequentist summary statistics (although P-values are very problematic and stopping rules are not really relevant). So personally, I think that there is room within the statistical menagerie for people who want to do EBM and those who want to do SBM without their having to snipe at each other. Stephen -----Original Message----- From: Evidence based health (EBH) [mailto:<mailto:[log in to unmask]>[log in to unmask]<mailto:[log in to unmask]><mailto:<mailto:[log in to unmask]>[log in to unmask]<mailto:[log in to unmask]>>] On Behalf Of Croft Daniel (RBF) NOC Sent: 08 November 2010 10:54 To: <mailto:[log in to unmask]> [log in to unmask]<mailto:[log in to unmask]><mailto:<mailto:[log in to unmask]>[log in to unmask]<mailto:[log in to unmask]>> Subject: Re: SBM-EBM shouting match Dear List, Would someone mind translating this debate into layman language for people like me who are new to statistics? Am I right in thinking that the people who argue for using Bayesian statistical methods want to consider what we know already when analysing new data. They do this by estimating values to represent that prior knowledge that then affect the analysis of the new data. The non-Bayesian people consider each new test with no prior considerations, so that any result is possible. And do these two camps generally split into Science-Based Medicine people and Evidence-Based Medicine people respectively? Regards, Daniel -----Original Message----- From: Evidence based health (EBH) [mailto:<mailto:[log in to unmask]>[log in to unmask]<mailto:[log in to unmask]><mailto:<mailto:[log in to unmask]>[log in to unmask]<mailto:[log in to unmask]>>] On Behalf Of Michael Power Sent: 08 November 2010 10:25 To: <mailto:[log in to unmask]> [log in to unmask]<mailto:[log in to unmask]><mailto:<mailto:[log in to unmask]>[log in to unmask]<mailto:[log in to unmask]>> Subject: SBM-EBM shouting match David Gorski at the Science-Based Medicine blog has posted a put-down of Steve Simon's defence of EBM: <http://www.sciencebasedmedicine.org/?p=8151> http://www.sciencebasedmedicine.org/?p=8151 I won't join this shouting match, or comment on the gratuitous ad hominems, or remark on variation of the "but Brutus is an honourable man" rhetoric, or draw attention (again) to the reification of EBM into a fantastic strawman. But, I do want to ask a serious question, provoked by Gorksi's comment that EBM ignores prior probabilities. There must be a reason why Bayesian methods are so seldom used. Professor Sir Michael Rawlins suggests five possibilities in his much lauded Harveian Oration (<http://bookshop.rcplondon.ac.uk/contents/pub262-9bc950aa-00e6-4266-8e80-e4bc63a25262.pdf>http://bookshop.rcplondon.ac.uk/contents/pub262-9bc950aa-00e6-4266-8e80-e4bc63a25262.pdf): 1) People "prefer the apparent (but illusory) security of a clear definition of what constitutes an 'extreme' result when tested against the null hypothesis, and they are reluctant to accept that either personal belief or judgement should come into play in decision making." "clinical investigators are much more reluctant to accept a subjective approach to the interpretation of probability than statisticians." 2) "there have been substantial controversies about the derivation of the prior probability" "too much has been made of the alleged difficulties" 3) "Bayesian analyses are computationally complex" 4) "some statisticians - albeit a dwindling number - are unfamiliar with the techniques of Bayesian analysis and are unwilling (or unable) to adapt. Some generously attribute this variation in skill-mix to a statistician's original choice of university. Others, less kindly, believe it to be generational. As one Bayesian explained to me: 'Statisticians who were taught how to use log books and slide rules can't usually do Bayesian statistics'" 5) "regulatory authorities have sometimes been hesitant to concede that Bayesian approaches may have advantages" These reasons, especially the 4th (I was taught a long time ago to use log books and slide rules), seem to me to be insufficient to explain why Bayesian methods are not more widely used. The question I want to ask is (at the risk of exposing my naiveté wrt Bayesian methods): do the proponents of Bayesian methods mistakenly assume that p-values and 95% confidence intervals indicate the true probability and true range of measurements? Well trained EBMers know that p-values and 95% CI-s are indicators of precision (statistical variation). P-values and 95% CI-s are not indicators of accuracy (truth). This requires other information, which is provided by critical appraisal and "triangulation" with other results. The way I understand it (or misunderstand it) is that with Bayesian methods the subjective assessment is done when the prior probability is chosen (note, it is not measured, even if a measurement is chosen). But, with frequentist methods, the subjective assessment is done when the risk of bias and error is assessed with critical appraisal. In other words, both methods require qualitative assessment of their results. I prefer the frequentist method because it makes transparent the essential role of critical appraisal to assess accuracy (risks of bias and error). Am I wrong? -- Carlos A. Cuello-García, MD Director, Centre for Evidence-Based Medicine Tecnologico de Monterrey School of Medicine Cochrane Collaboration Iberoamerican branch CITES piso 3. Morones Prieto 3000 pte. Col. Doctores 64710 Monterrey, NL. Mexico. ? +52.81.8888.2223 & 2154. Fax: +52.81.8888.2148 Skype: dr.carlos.cuello <http://www.cmbe.net>www.cmbe.net<http://www.cmbe.net><<http://www.cmbe.net>http://www.cmbe.net> ? Twitter<<http://twitter.com/CharlieNeck>http://twitter.com/CharlieNeck> ? Linkedin<<http://mx.linkedin.com/in/drcuello>http://mx.linkedin.com/in/drcuello> The content of this data transmission must not be considered an offer, proposal, understanding or agreement unless it is confirmed in a document signed by a legal representative of ITESM. The content of this data transmission is confidential and is intended to be delivered only to the addressees. Therefore, it shall not be distributed and/or disclosed through any means without the authorization of the original sender. If you are not the addressee, you are forbidden from using it, either totally or partially, for any purpose -- Carlos A. Cuello-García, MD Director, Centre for Evidence-Based Medicine Tecnologico de Monterrey School of Medicine Cochrane Collaboration Iberoamerican branch CITES piso 3. Morones Prieto 3000 pte. Col. Doctores 64710 Monterrey, NL. Mexico. ? +52.81.8888.2223 & 2154. Fax: +52.81.8888.2148 Skype: dr.carlos.cuello www.cmbe.net<http://www.cmbe.net> ? Twitter<http://twitter.com/CharlieNeck> ? Linkedin<http://mx.linkedin.com/in/drcuello> The content of this data transmission must not be considered an offer, proposal, understanding or agreement unless it is confirmed in a document signed by a legal representative of ITESM. The content of this data transmission is confidential and is intended to be delivered only to the addressees. Therefore, it shall not be distributed and/or disclosed through any means without the authorization of the original sender. If you are not the addressee, you are forbidden from using it, either totally or partially, for any purpose CoxHealth – a Top 100 Integrated Health Care Network COXHEALTH CONFIDENTIALITY NOTICE: This e-mail message, including any attachments, is for the sole use of the intended recipient(s) and may contain confidential and privileged information protected by law. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply e-mail and destroy all copies of the original message.