Dear Arri
we had the same problem when we were putting together a guideline on chronic
leg ulcers. When it came to choosing a grading system, the Oxford CEBM
system is the most comprehensive and does have the caveat that allows for
dropping the grade of the recommendation despite the level of evidence.
However, we wanted a simpler system that made provision for diagnostic
studies as well as introducing a caveat, so adapted the AHCPR system and
added a caveat (a * beside the level of evidence) that dropped the grade of
the recommendation to C (consensus opinion).
The caveat came into play where
(1) systematic reviews did not find any evidence or where there was a too
little evidence to draw a definitive conclusion e.g. the systematic review
on debridement in venous leg ulcers found no RCTs comparing debridement with
no debridement with healing as an outcome, and therefore could not make any
conclusion.
(2) where there was an inconclusive RCT, usually one that said there was no
difference between treatments (when we calculated the confidence intervals
by us - usually these studies were at risk of Type II error). You might ask
why include these studies, and I guess that comes down to what is the
purpose of a guideline. In my view a guideline should communicate both
certainty and uncertainty, and in some areas of wound care there is very
little [conclusive] evidence. Therefore we chose to include the evidence and
downgrade the recommendation e.g. a small RCT comparing ulcer healing at 20
days (short timeframe) in a group routinely treated with antibiotics and
group not treated with antibiotics - no significant difference in healing
rates, and it is the only piece of evidence we have.
(3) where the evidence had been extrapolated from another area e.g. a CCT
that compared acute lacerations cleansed with either tap water or sterile
saline. We used this as evidence to support cleansing/bathing ulcers with
tap water.
But in all the above cases the caveat came into play and we graded the
recommendation Level C.
As you indicated, when we reviewed other guidelines' approach to grading
recommendations, none seemed to introduce a caveat for the "grey areas".
If you wanted an electronic copy of our guideline, I would be happy to
forward it to you (or it will shortly be available from the New Zealand
Guidelines Group's website www.nzgg.org.nz).
regards
Andrew Jull
> -----Original Message-----
> From: Dr A Coomarasamy [SMTP:[log in to unmask]]
> Sent: Thursday, 5 October 2000 07:19
> To: [log in to unmask]
> Subject: A systematic review without sufficient power.
>
>
> Dear Friends,
>
> I often see Grade A recommendations being generated from level 1a evidence
> (good SR's with homogeneity etc). When I stated to closely examine the
> evidence myself, I came across several SRs (which have led to grade A
> recommendations in various professional body guidelines) that just did not
> have the aggregate sample size (power) to answer the question that the SR
> set out to examine. An SR with an aggregate sample size of a hundred or so
> is no better than a primary study that unsuccessfully attempts to study a
> rare outcome with small numbers.
>
> What is the solution? Whatever it is, it has to be explicitly reflected on
> the levels of evidence and grades of recommendations - as many do not
> question the evidence that generates grade A recommendations and accept it
> as the gospel truth.
>
> I do note that Sackett in his 'how to practice and teach EBM' (page 176,
> 2nd Ed) does mention (although only in fine print) that SRs with
> troublesome heterogeniety and insufficient precision should generate grade
> D recommendation! - I don't think that this is being practiced widely. (
> He also suggests attaching a minus sign to indicate this problem
> heterogeniety etc).
>
> arri
>
>
> -----------------------------------------
> A Coomarasamy
> Tel 0956 498 457
> [log in to unmask] <mailto:[log in to unmask]>
> -----------------------------------------
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
|