Andrew
Thank you for a thought-provoking question.
Why would you want to meta-analyse inter-observer reliability data across
studies? I would see this type of reliability data as a quality threshold
variable that each study needs to meet, rather than something to combine
across studies.
But on the other hand, it may be that I have misunderstood what is your
question for part 2a of the review. This could make all the difference to the
answer!
Would be keen to hear other views.
Dr Frances Gardner
Director of Graduate Studies in Evidence-Based Social Work
Department of Social Policy & Social Work,
Oxford University
32 Wellington Square, Oxford OX1 2ER, UK
tel:44-1865-270334 [log in to unmask]
Andrew Booth <[log in to unmask]> writes:
> Excuse me if this sounds a fundamental question. I have a student who I am
> supervising for a dissertation - it is a diagnostic question so I am happy
> on the systematic review part but am on new ground when it comes to the
> meta-analysis.
>
>
>
> The student's review is in two stages. Phase1 of the review, evaluating
> validity, compares low-tech instruments in
>
> comparison to the gold standard, which is x-ray. The outcomes tend to be
> reported in degrees and the studies have inferred validity or not through
> the use of Pearsons correlation coefficient, LoA, ICC amongst others.
>
>
>
> The studies included in Phase 2 of the review, evaluating reliability - both
> inter and intra observer, tend to use Kappa or ICC.
>
>
> We believe that meta-analysis of the first phase is (fairly!)
> straightforward [although would welcome any suggestions or methodologic
> papers or examples].
>
>
>
> However the statistical advice we have sought has said re: phase 2 "In
> practice, I am not sure that methods for combining ICCs or Kappa statistics
> actually exist. (You may be able to treat the ICC or Kappa value as if it
> were a mean value and use methods for combining the mean outcomes)." So that
> is why I am opening it up to the wider world.
>
> Anyone able to answer this, please?
>
>
>
> Andrew Booth
>
> Director of Information Resources and Senior Lecturer in Evidence Based
> Healthcare Information
>
>
>
--
|