Dear list,
These are questions from a colleague. (There are two). Your time and
help are valued.
PART I
I have not dealt with kappa for a while so refresh my memory. We have
data of 2 by 2 agreement:
YY 4217
YN 54
NY 20
NN 1
So percentage of same response is high (4218/4292 ~ 98.3%), but kappa
is
lousy (0.0194). This is presumably because only one response was NoNo
(NN), which is rare compared to the Yes/No responses.
So my feeling is to report the percentage of same response and the
kappa,
and just describe why the kappa is so low.
Any other comments or insights into this wonderful 2 by 2 table??
PART II
I guess the reason my kappa question seemed odd is the nature of what
we're using kappa for, which is probably questionable. We have an
on-line patient registry with several thousand patients. Now that the
data has accumulated, the investigators (who thought at first no one
would participate), now want to assess the "quality" and "validity" of
the data. We have been scratching our heads about how to deal with
it, and one way was assessing the internal consistency of the data.
The 2 by 2 table I posed before was basically "Did the patient have
the disease" and "Did the patient get treated for the disease"
questions. So getting a lot of Yes/Yes was good, even though the
kappa was poor. We have other combinations of questions where we
would expect a better distribution of Yes/No results
Do any of you have other thoughts about how to deal with this question
of
validity of an existing database?
THANKS AGAIN!!!
* * *
Brett Larive [log in to unmask]
Dept of Biostatistics/Wb4 phone: 216-444-9925
Cleveland Clinic Foundation fax: 216-445-2781
9500 Euclid Avenue
Cleveland, OH 44195
|