while waiting for more authoritative responses, i'll offer some.
1. anybody who says there is a single, correct statistical analysis or
appraoch is a mountebank.
2. i reckon that with non-stat sig results the NNT & NNH are even more
important than when the results ARE stat sig. they tell us whether the
results include or exclude clinically important differences. for example,
when we did an RCT of ec-ic bypass for threatened stroke, it wasn't that
the outcomes were stat non-sig different in the surgical and medical arms
that convinced us and our surgical colleagues to abandon the operation
(its performance declined by 80% within about 18 months). it was that
the 95% conf interval around the observed difference EXCLUDED any
clinically important benefit.
3. i tell myself to remember that an NNT of 20 with 95% conf intervals
of +40 to -20 means that the NNT could be as low as 40 but as high as
infinity, and that the NNH could be as low as 20 but also as high as
infinity (that is, you don't get from +40 to -20 by going through zero,
you get there by going the other way, through infinity.
hope this keeps folks amused until the real experts respond.
cheers
dls
ps: if you want me to join you in an appeal to the journal, i'd be happy
to take up your cause.
On Tue, 22 Sep 1998, Atle Klovning wrote:
> Date: Tue, 22 Sep 1998 22:50:36 +0200
> From: Atle Klovning <[log in to unmask]>
> To: Dave Sackett <[log in to unmask]>,
> EBH Discussion Group <[log in to unmask]>
> Cc: Douglas Altman <[log in to unmask]>
> Subject: NNTs/NNHs on non-significant differences in event rates
>
> Dear list members,
>
> we tried to write a correspondence letter where we calculated NNTs for the results presented in The Lancet on the HOT study (what is the optimum diastolic target blood pressure?)
>
> We found that by decreasing the target blood pressure from 90 to 80 mm Hg for 3.8 years in 180 non-diabetics would cause one death (total mortality), while treating 39 diabetics would prevent one death. The differences in the event rates were however not significant.
>
> Since the differences in outcomes were not significant, a referee strongly disagreed with our converting the original table into NNTs. We were "turned down" by the comment that if there was no significant differences in the outcomes, we would have to settle with "the fact that there were no difference in effects" and calculating NNTs on non-signiificant differences would be a statistical violation.
>
> My question is, after looking at different papers/books/websites on how to calculate NNTs and Altman's comments, whether this is a statistical violation or not? Is it really possible to calculate NNTs on non-significant differences?
>
> We look forward to Altman's paper-to-come in the BMJ.
>
> Atle Klovning, MD
> Specialist in general practice/family medicine
> Research Fellow
> Division for General Practice
> University of Bergen
> Ulriksdal 8C, N-5009 Bergen,
> Tel (+47) 55 58 61 37 *** Fax (+47) 55 58 61 30
> http://www.uib.no/isf/people/atle/atle.htm
>
>
>
> Fra: Dave Sackett <[log in to unmask]>
> Til: EBH Discussion Group <[log in to unmask]>
> Kopi: Douglas Altman <[log in to unmask]>
> Dato: 22. september 1998 16:55
> Emne: Re: 95% CI's for NNTs (fwd)
>
>
> >here is a response from doug altman, director of the centre for statistics
> >in medicine here in oxford, and a member of our centre.
> >cheers
> >dls
> >............................................................................
> >Prof David L. Sackett
> >Director, NHS R&D Centre for Evidence-Based Medicine
> >Consultant in Medicine Editor, Evidence-Based Medicine
> >Nuffield Department of Medicine, University of Oxford
> >Level 5, John Radcliffe Hospital, Oxford OX3 9DU, England
> >Phone: +44-(0)1865-221320 Fax: +44-(0)1865 222901
> >Email: [log in to unmask] WWW: http://cebm.jr2.ox.ac.uk
> >............................................................................
> >Doug Altman writes:
> >
> >Iain Buchan is right to draw attention to the improved formula developed
> >recently by Newcombe for the CI of the difference between two proportions.
> >This is a valuable advance, but the old (standard) formula works fine
> >except when the either the numbers of events or the observed proportions
> >are very small (or very near n or 1 respectively) - i.e it works most of
> >the time. I doubt whether the problems encountered by Tony Redmond are
> >related to this issue.
> >
> >The primary reason for the discrepancies noted by Tony is likely to be the
> >fact that the formula given on the Bandolier web page is wrong (see PS
> >below). The simplest approach to getting the CI for the NNT is to calculate
> >the CI for the absolute risk difference (the difference between the
> >observed proportions) - which can be done using a simple formula found in
> >any good statistics book - and take reciprocals of these two values.
> >
> >I might add that the Bandolier web page also says that when the difference
> >between the groups (the treatment effect) is not significant, then it is
> >'unwise to bother with the CI for the NNT'. I disagree - CIs are especially
> >useful when there is a so-called negative result, especially given that
> >trials tend to be too small to have adequate power to detect useful benefits.
> >
> >@@@@ dls note: that's why, in the CATMaker, we calculate and display both
> >NNTs (e.g., 20 to infinity) and NNHs (e.g., 50 to infinity), recalling
> >that the "NN" when stated 20-50 gets between these two umbers via infinity
> >@@@@
> >
> >The usual reason for not giving CIs for the NNT in such cases is that the
> >results may seem somewhat nonsensical. I have a paper due out soon in the
> >BMJ explaining how to calculate and present such confidence intervals even
> >when the treatment effect is not significant.
> >
> >In this paper I also note that the term 'number needed to harm' (NNH) is
> >confusing and grammatically incorrect, and suggest that we use instead of
> >NNTB and NNTH - the number need to treat to benefit (or harm).
> >
> >Doug Altman
> >
> >PS The error in the formula on the Bandolier web page quoted is that there
> >should be yet another set of brackets around everything after 1/ at the
> >beginning. But the way of calculation given above is much simpler to follow.
> >
> >_________________________________________________
> >Douglas G Altman Tel: +44 (0)1865 226799
> >ICRF Medical Statistics Group Fax: +44 (0)1865 226962
> >Centre for Statistics in Medicine
> >Institute of Health Sciences
> >Old Road, Headington email: [log in to unmask]
> >Oxford OX3 7LF, UK http://www.ihs.ox.ac.uk/csm
> >_________________________________________________
> >
> >
> >
> >
>
>
---------------------------------------------------------------------------
Prof David L. Sackett Email: [log in to unmask]
Director, Phone: +44-(0)1865-221320
Centre for Evidence-Based Medicine Fax: +44-(0)1865-222901
Level 5, John Radcliffe Hospital WWW: http://cebm.jr2.ox.ac.uk/
Oxford OX3 9DU
England
---------------------------------------------------------------------------
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
|