 |
 |
it's an ongoing issue with understanding and we've done poorly so far not educating enough our colleagues about 95%CI and it's usefulness in medical research. as a result, P-value use, misuse and abuse will rule the show. We've been taught that 95%CI of a measurement means that if the study is replicated 100 times, 95 results will be within the interval. We've also been taught that the width of the interval is a relative measure of precision of the estimate and matters a lot in meta-analysis. We also learned, that the limits of 95%CI can be used in sensitivity analyses. For instance, one study of biomarker showed point estimate of RR of death increased x40 (95%CI 1.2-99), it mean to me that the estimate is grossly imprecise and I can’t rely on this study results, but I can design a future study making my entry assumption that the effect size could "as much as x40" for my sample size planning, if no more precise information is available. Nick Nickolas Myles, MD, PhD, MSc, FRCPC Anatomical pathologist, St.Paul’s Hospital, Clinical Associate Professor, University of British Columbia Department of Pathology and Laboratory Medicine 1081 Burrard St, Vancouver, BC, V6Z1Y6 Phone (604) 682-2344 x 66038 Email: [log in to unmask] From: Evidence based health (EBH) [mailto:[log in to unmask]] On Behalf Of Mohammed T. Ansari Sent: Wednesday, August 26, 2015 2:47 PM To: [log in to unmask] Subject: Re: Use of software for obtaining confidence intervals James and Ben: my problem is that I find the explanation "IT MEANS in a very large number of repetitions of the study, 95% of all CIs obtained will contain the “true” value of the treatment effect in the population studied" incomprehensible. The fact is that I have conducted one study and computed a point estimate and its 95% CI. Now the hypo that if I repeat the study a thousand times and compute their CIs, 950 of the computed CIs will contain the population parameter provides no meaningful info to me because I do not know what those 950 CIs are (and to what degree they overlap or do not overlap with the CI at hand)...what I know is this CI that I have in front of me. What can I conclude from the interval I have? I think a good answer has to be philosophical rather than mathematical.....can we rephrase this in some other way? On Wed, Aug 26, 2015 at 4:07 PM, McCormack, James <[log in to unmask]<mailto:[log in to unmask]>> wrote: Thanks Ben - glad you agree that I’m on the right track - it is amazing how many different interpretations we have of confidence intervals - I’m now 95% confident that what I said was correct :) On Aug 26, 2015, at 1:00 PM, Djulbegovic, Benjamin <[log in to unmask]<mailto:[log in to unmask]>> wrote: I agree the statement: "The values in between these lines indicate the range of values within which we are 95% certain (or confident, hence the term confidence intervals) that the true value is likely to lie” is wrong. When we calculate a 95% confidence interval we do not calculate a “true” assessment about this particular interval. We only say that if we repeat our study thousands of times, 95% of the intervals we computed will contain the true value from x1% to x2%. This does not mean that the true percentage is between x1% and x2%. The frequency with which this single computed 95% CI contains the true value is either 100% or 0%. (This is akin to when we say a response rate is 40% - 4 out 10 patients will respond to Rx, but in every individual patient the response will be either zero or 100%). Ben Djulbegovic From: Evidence based health (EBH) [mailto:[log in to unmask]] On Behalf Of McCormack, James Sent: Wednesday, August 26, 2015 12:13 PM To: [log in to unmask]<mailto:[log in to unmask]> Subject: Re: Use of software for obtaining confidence intervals Interesting take but I don’t think I’m doing that (or at least I didn’t mean to). What I was trying to define was what a single confidence interval from a single study means in the context of if one did 100 similar studies and got 100 different point estimates and confidence intervals. I think the bottom line is that all you can say from a single confidence interval is that it represents a plausible range of values for the effect. I hear people say it means that we are 95% confident that the result lies within the range of a single CI. To quote an evidence-based text "The values in between these lines indicate the range of values within which we are 95% certain (or confident, hence the term confidence intervals) that the true value is likely to lie” which I think is wrong. Although I would love to be enlightened if I’m completely off base. Hope that makes sense. James On Aug 26, 2015, at 8:10 AM, Mohammed T. Ansari <[log in to unmask]<mailto:[log in to unmask]>> wrote: Thanks James. Just one question given your definition of 95% CI "IT MEANS in a very large number of repetitions of the study, 95% of all CIs obtained will contain the “true” value of the treatment effect in the population studied (assuming random sampling)": - how can one use the same term to define the term one is defining (re" your "95% of all CIs" to define a CI)? ____________________________________ Mohammed T. Ansari about.me/MTAnsari<http://about.me/MTAnsari> On Wed, Aug 26, 2015 at 10:25 AM, McCormack, James <[log in to unmask]<mailto:[log in to unmask]>> wrote: Having taught EBHC for 25 years one of the things I see is that medical graduates don’t understand what confidence intervals mean and do not mean - I think more importantly than being able to calculate a CI they need to understand the context and what it means to research findings. The way I understand confidence intervals is 1) IT MEANS in a very large number of repetitions of the study, 95% of all CIs obtained will contain the “true” value of the treatment effect in the population studied (assuming random sampling) 2) IT DOES NOT MEAN the probability that the “true” magnitude of the effect lies within the range observed in a given study 3) IT DOES represent a plausible range of values for the effect – not a probability of its magnitude Happy to hear if I am misinterpreting the concept We recently wrote an article showing how authors confuse the meaning of confidence intervals. This group may find it of value as we provide a number of examples (statins and mortality in primary prevention) where authors misinterpret their results because they misinterpret what a confidence interval means and does not mean. http://www.ncbi.nlm.nih.gov/pubmed/24172248 James McCormack On Aug 25, 2015, at 5:05 AM, Margaret MacDougall <[log in to unmask]<mailto:[log in to unmask]>> wrote: Hello As a medical statistician, I have strong support for the use of confidence intervals in the reporting of clinical research findings. However, I am also conscious that medical graduates may lose their access to licensed software previously used in their undergraduate learning to support such activities. For example, both Minitab and Confidence Interval Analysis are possible choices for non-specialists and often supported by University licenses. However, do medical graduates have an incentive to obtain their own licenses for such software on entering the workplace? I would be interested to learn from medical graduates as to their personal choices in the above context and indeed, as to whether or not such persons have found freeware sufficiently reliable and easy to use to adopt in their own working practices specifically for obtaining confidence intervals (not for statistics more generally). This query is also of relevance to the management of courses for distance learners, not all of whom may have access to a licensed statistical package for calculating confidence intervals but still need to perform the calculations to meet programme requirements. Many thanks in advance Best wishes Margaret -- ____________________________________ [https://d13pix9kaak6wt.cloudfront.net/signature/me-badge.png] Mohammed T. Ansari about.me/MTAnsari