Hello all.
Many thanks to all of you who took time to respond to my query regarding the
use of Kappa with multiple raters. Before my summarise of the information I
received I have another query. Have any of you been able to use the macro,
which is available from the SPSS homepage, for calculating Kappa with more
than two raters. I have downloaded it but am not having much joy?
Most of you agreed that it was simply a matter of adapting the original
formula and various explanations of this were cited. Most of you suggested
some version of Fleiss. The most popular was -
Fleiss, J.L.,(1981). Statisitcal methods for rates and proportions.
Additional suggestions were:
- Franklin, R., Allison, D, and Gorman, B. (1997). Design and analysis
of single case research. Lawrence Erlbaum.
>From Graeme Hutcheson & Nick Sofroniou
- Integration and generalization of kappas for multiple raters.
Conger,A.J. Psychological Bulletin. 1980 Sep; Vol 88(2): 322-328
- Measuring nominal scale agreement among many raters. Fleiss, J.L.
Psychological Bulletin. 1971 Nov; Vol. 76(5): 378-382.
- Measures of response agreement for qualitative data: Some generalizations
and alternatives. Light, R.J. Psychological-Bulletin. 1971 Nov; Vol. 76(5):
365-377.
>From Wil Irwin
- Banerjee M Beyond kappa: A review of interrater agreement measures
CAN J STAT 27: (1) 3-23 MAR 1999
- POSNER KL, SAMPSON PD, CAPLAN RA, et al. MEASURING INTERRATER RELIABILITY
AMONG MULTIPLE RATERS - AN EXAMPLE OF METHODS FOR NOMINAL DATA STAT MED 9:
(9) 1103-1115 SEP 1990
>From Mitchum Bock
Thanks again,
Michael.
---------------------------------
Michael Gormley
Department of Psychology
University of Dublin
Trinity College
Dublin 2
email: [log in to unmask]
---------------------------------
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
|