I have posted this request (but received no definitive answers) on two other lists so apologies for possible cross-posting.
I have several sources that suggest it is possible to extend weighted Kappa or Pi to the situation of more than two raters but none that give an explicit method. Both Siegel & Castellan* and Fleiss** give a cIear exposition of the calculation for the unweighted multirater case and numerous sources explain the weighted birater case. I would be grateful therefore if anyone can point me towards a text or paper that details the method/formula for calculating a weighted multirater version of either Cohen's Kappa or Scott's Pi and, if possible, gives a formula for the variance.
Thanks in advance for any advice,
* Siegel, S. and N. J. Castellan, Jr (1988). Nonparametric Statistics for the Behavioural Sciences. New York, McGraw-Hill.
** Fleiss, J (1981) Statistical Methods for Rates and Proportions. New York, Wiley
Research Fellow - MMC
Anaesthesia Recruitment Validation Group
Peninsula Medical School / SW Peninsula Deanery