Dear All,
Thanks for those valuable comments. I guess what went
wrong in the first place was the two datasets, from
Muller's (stat med, 1994)and Bland's (Lancet,
1986)papers, I tested out by hand. According to
Bartko's paper (psychological bulletin, 1976), the ICC
for a mixed effect model is =
(MSB-MSW)/(MSB+[c-1]MSW).
Muller's data set yielded ICC=0.918 and Pearson's r
=0.924 and Bland's data set gave ICC=0.9429, Pearson's
= 0.943. All these results, I believe, were totally
coincidental.
Below is a summary of comments and references. Again
many thanks for the clarification.
Astride
The similarities and differences are tackled by
Streiner and Norman book, "Health Measurement Scales"
(Oxford Univ Press). ISBN 0-19-262670-1 (2nd edition)
'Reliability of isokinetic ankle dorsiflexor strength
measurements in healthy young men and women.'
Holmback, Porter, Downham, Lexell Scand J Rehab Med
31: 229-239, 1999
"Effects of model misspecification in the estimation
of variance components and intraclass correlation for
paired data" Robert H. Lyles Statistics in Medicine,
vol 14,1693-1706 (1995)
======================================================
I think that your calculations may be wrong. Try
these
data:
Sub x1 x2
-----------
1 12 14
2 13 14
3 14 15
4 13 16
5 15 15
6 16 17
7 17 16
8 18 17
9 16 15
10 15 14
Using Stata, I get ICC = 0.5982, r = 0.6666.
It is not clear what kind of agreement problem you are
trying to solve.
In the study of reliability, i.e. how well
observations by
the same method agree, the ICC is appropriate because
it
does not take the ordering of the observations into
account. These are regared as equivalent replicates,
and
so the order does not mean anything and should not be
used
in the calculation.
In the study of two different methods of measurement,
the
order (i.e. method A or method B) is clearly very
important
and ICC is not appropriate.
The "Bland-Altman" method is designed to answer the
question: to what extent can measurements obtained by
the
two methods be used interchangeably. Correlation is
clearly irrelevant to to this question.
================================================
Correlation is still only a measure of how well
observations fit a
straight line. It doesn't assess whether the slope of
the line is 1.0
or some other value. Agreement would usually require
the former rather
than the latter.
It isn't usually obvious which of two variables is
dependent and which
independent when agreement or reliability is the
issue. Moreover both
variables usually have measurement error under these
circumstances
which
leads to a bias toward zero in regression
coefficients. Thus, doing
both regressions, y on x and x on y, can give results
that are quite
different.
It is most useful to try to get an estimate of the
measurement errors
in
y and x. The Bland-Altman method seems to work quite
well despite
little theory. It makes sense at least.
A bootstrap estimate of variance also makes sense.
====================================================
Sorry to disillusion you but it is very easy to design
a data set with
a high Pearson Correlation but a poor ICC.
Have a look at the data set below which I created this
morning:
x1 x2
86.35 160.05
99.44 296.32
95.87 183.93
94.59 200.52
72.16 82.48
73.49 309.48
95.24 198.65
124.51 504.93
97.20 236.48
95.25 169.93
94.89 224.05
82.59 393.57
121.41 389.42
86.92 291.49
93.29 339.57
73.22 234.60
77.14 206.65
115.20 329.59
111.03 406.04
108.76 230.43
It has a significant Pearsons correlation but its ICC
is not
significant at all. If you want to know why I think
you should read
carefully what an ICC corrects for.
Sorry to disillusion you but it is very easy to design
a data set with
a high Pearson Correlation but a poor ICC.
Have a look at the data set below which I created this
morning:
x1 x2
86.35 160.05
99.44 296.32
95.87 183.93
94.59 200.52
72.16 82.48
73.49 309.48
95.24 198.65
124.51 504.93
97.20 236.48
95.25 169.93
94.89 224.05
82.59 393.57
121.41 389.42
86.92 291.49
93.29 339.57
73.22 234.60
77.14 206.65
115.20 329.59
111.03 406.04
108.76 230.43
It has a significant Pearsons correlation but its ICC
is not
significant at all. If you want to know why I think
you should read
carefully what an ICC corrects for.
____________________________________________________________
Do You Yahoo!?
Get your free @yahoo.co.uk address at http://mail.yahoo.co.uk
or your free @yahoo.ie address at http://mail.yahoo.ie
|
