I agree. As an example the SIGN Guideline on leg ulcers gave a Grade A
(Level 1b evidence - evidence obtained from at least one randomised
controlled trial) to the recommendation that tap water can be used to
cleanse leg ulcers. However, the study pseudo-randomisation (alternate
weeks), the result was sensitive to loss, and the evidence was extrapolated
from a study population with acute lacerations treated in ED, not leg
ulcers. In the presence of alternative studies, this study could be
excluded, but it is the only study that has compared irrigation of wounds
with sterile saline vs tap water. If included it would receive a Grade D
recommendation using the Oxford hierarchy.
What the above example illustrates I think is that ultimately the integrity
of an grading of a recommendation is reliant on the skill of those
undertaking the critical appraisal.
Another criticism of the SIGN system is that it does not address the issue
that different clinical questions require different study designs. The SIGN
system seems to assume either that:
1. guidelines only address issues of treatment, or
2. that the RCT or systematic review of RCTs are the gold standard for all
types of clinical questions.
Patently both are absurd. Guidelines address issues of diagnostic testing
for instance. A cohort study is going to provide the necessary information,
but SIGN would only ever give a well designed cohort study of a diagnostic
test a B grade recommendation.
Andrew Jull
Clinical Nurse Consultant
Auckland Hospital
NEW ZEALAND
> -----Original Message-----
> From: Djulbegovic, Benjamin [SMTP:[log in to unmask]]
> Sent: Wednesday, 15 August 2001 23:44
> To: [log in to unmask]
> Subject: Re: validated instruments for critical appraisal & levels of
> recommendation
>
> Another point here-and I think the crucial one- is that (grades of)
> recommendations are directly linked to a level of evidence system, which
> means different recommendations will be derived depending on the hierarchy
> of study types used. For example, SIGN researchers may end up with
> different
> grades of recommendations if they used Oxford ranking scheme instead of
> "old" AHRQ system. Ditto goes for the instruments for critical appraisal-
> many have been developed, but I am yet to see a validated one taking into
> account empirical evidence associated with a given methodological
> dimension
> (instead of normative beliefs of a good trial design). (SIGN folks claim
> that they did validate their checklists; unfortunately their instrument
> appears to be available only to the members of the group or on special
> request).
>
> ben d
>
>
|