Dear all,
I've received a lot of replies to my query on CI for sensitivity and specificity.
Below you can find the summary.
To
Prof. M.Bland, Iain Buchan, Prof. Richard Lowry, Robert Newcombe, Aristeu Vieira da Silva, Francois Harel, Steffen Witte, Max Bulsara, Lesley Bailey,
Zoann Nugent, Alicja Rudnicka:
Many thanks for your helpful suggestions.
Prof. Martin Bland:
These are simple binomial proportions. However, as they are often close
to 100% and based on small numbers, the large sample Normal aproximation
breaks down. There are several methods to produce better confidence
intervals and these will always be asymmetrical because of the limit at
100%. I prefer the method using the exact probablities of the Binomial
distribution. You find the value of p which would give the probability
of the number of successes to be that observed or more to be 0.025, and
the value of p which would give the probability of the number of
successes to be that observed or less to be 0.025. This two values of p
are the confidence interval. I have written an MSDOS program which does
this, downloadable from my website, address below.
http://www.sghms/ac/uk/depts/phs/staff/jmb/jmb.htm
Martin
-----------------------------------
Iain Buchan:
Luigi,
Take a look at the diagnostic test function under Analysis_Miscellaneous
in StatsDirect from http://www.statsdirect.com/update.htm .There are
full referencences under the help system there.
Any properly calculated binomial confidence interval can be
asymmetrical.
Kind regards
Iain Buchan
-----------------------------------
Prof. Richard Lowry:
You'll find a calculator for these items on the site
http://faculty.vassar.edu/lowry/VassarStats.html
under 'Clinical Research Calculators', then 'Clinical Calculator1.
--Richard Lowry
-----------------------------------
Robert Newcombe:
Sensitivity and specificity are simply proportions. The simplest
(Wald) CI formula for a proportion p = r/n, viz. p +/- z*sqrt(p*(1-
p)/n), doesn't work at all well. The problems can really be traced
to its symmetry. (See my first 1998 paper below). Numerous much
better methods have been proposed. One of the best is due to Wilson
(1927). It works very well whether n is large or small, and whether
r or n-r is large, small or even zero. It is implemented in the
first block of the accompanying Excel spreadsheet.
If you want a CI for the difference in sensitivity between 2
different tests, the 3rd block of the spreadsheet (paired case) lets
you do this, assuming that all subjects get the gold standard test G
and both the tests A and B to be compared. If some subjects get A
and G, and others get B and G, use the 2nd block (unpaired case).
However, it makes more sense to compare sensitivity and specificity
*simultaneously* between the 2 tests A and B. I have a graphical
method to do this, which works both in the paired case and the
unpaired case. My 2001 paper below describes the paired case
References.
Newcombe RG. Two-sided confidence intervals for the single
proportion: comparison of seven methods. Statistics in Medicine 1998,
17, 857-872.
Newcombe RG. Interval estimation for the difference between
independent proportions: comparison of eleven methods. Statistics
in Medicine 1998, 17, 873-890.
Newcombe RG. Improved confidence intervals for the difference
between binomial proportions based on paired data. Statistics in
Medicine, 1998, 17, 2635-2650.
Newcombe RG. Simultaneous comparison of sensitivity and specificity
of two tests in the paired design: straightforward graphical
approach. Statistics in Medicine 2001, 20, 907-915.
Hope this helps.
Robert Newcombe.
--------------------------------------------
Aristeu Vieira da Silva:
Dear Dr. Santoro,
The paper: Mackinnon, A. (2000) A spreadsheet for the calculation of comprehensive statistics for the assessment of diagnostic tests and inter-rater agreement. Computers in Biology and Medicine, 30(3), 127-134, maybe help ... the author propose a spreadsheet and I attach this at this e-mail ...
Aristeu Vieira da Silva
--------------------------------------------
Francois Harel:
Hi,
In Joseph L. Fleiss (1981). Statistical for rates and proportions. John Wiley & Sons, Inc. NY.
Fleiss gives 2 formulae for the CI of a proportion.
Formula 1.29 the CI (with continuity correction) is symetric.
Formula 1.26 (or 1.27) is considered the exact CI for a proportion and is asymetric.
I hope that it will help.
Best regards,
--------------------------------------------
Steffen Witte:
Dear Luigi,
e.g. normal CI for binominal (asymptotically or exact)
\bibitem{DALY1992} Daly, L. (1992). Simple SAS macros for the calculation
of exact binomial and Poisson confidence limits. \textit{Comput. Biol.
Med.}, \textbf{22}(5), 351-361
sas-macro cibinom.sas available at
http://ftp.urz.uni-heidelberg.de/ftp/pub/sas/makros
or
Hilgers RA (1991) Distribution free confidence bounds for ROC curves.
Methods Inform. Med., 30, 96-101
Best regards from Heidelberg.
-------------------------------------------
Max Bulsara:
Hi Luigi
It's a Binomial proportion. The following www site will work out exact Binomial CI: http://www.members.aol.com/johnp71/javastat.html#Confidence
Hope it helps
Max
--------------------------------------------
Lesley Bailey:
Dear Luigi,
I found and used a formula which gives asymmetric confidence intervals. It relates to proportions and their differences and is described in Chapter 6 of the book below. There is a section in the same book dealing with sensitivity & specificity. The book is: 'Statistics with confidence, 2nd edn' edited by Douglas Altman, David Machin, Trevor Bryant & Martin Gardner. It is published by BMJ Books, and the 2nd Edition is from 2000. I have found it to be very useful, and also easy to use. I hope that this is useful to you.
Best wishes,
Lesley
---------------------------------------------
Zoann Nugent:
Unless p=.5, confidence intervals on
proportions are always asymetrical. It
only becomes important for p approaching
0 or 1 and small sample sizes.
Any advanced stats book should give the equations,
based on the F distributions.
-------------------------------------------------------
Alicja Rudnicka:
the following article, or its reference list, may be of use
Harper & Reeves, Reporting of precision of estimates for diagnostic
accuracy:a review BMJ 1999:318;1322-3
|