Dear Bob, Paul, and Arri
Thank you very much for your valuable inputs and clarification to this
matter and sorry for the late reply as Thursday & Friday are the week end in
this part of the world and I was in a short vacation.
So we can assume that a cross sectional blind comparison with a reference
standard tests the test ACCURACY in the optimal conditions "for the test"
and that RCTs test the test PERFORMANCE "in a clinical setting". Something
analogous to the efficacy and effectiveness of therapy.
If the above is convincing to you, I would like to add that as both aspects
are important from the clinician point of view this should be clarified
somehow in the hierarchy of searching for evidence for diagnosis as what is
interpreted from this hierarchy is that RCTs can not be used in the
evaluation of diagnostic tests at all rather than can be used in certain
situations for a diagnostic test.
Perhaps we might improve the hierarchy by splitting it into two: one to test
for "test accuracy" in ideal situations (the present one that can be
accordingly re-named) and to add another hierarchy for "diagnosis" in
general that can start with RCTs on the very top of the hierarchy.
We might also consider to design a special appraisal sheet/tool for RCTs
dealing with diagnosis.
Best of wishes and thank you very much,
Abdelhamid
Prof. Dr. Abdelhamid Attia
Prof. of Ob & Gyn, Cairo University
President; Arab Federation of EBM
----- Original Message -----
From: "Arri Coomarasamy" <[log in to unmask]>
To: <[log in to unmask]>
Sent: Thursday, May 04, 2006 10:22 PM
Subject: Re: Evidence for diagnostic tests from RCTs?
Dear Abdelhamid,
RCTs of tests are fraught with difficulties of their own.
For a good analysis of this, please see:
Bossuyt PM, Lijmer JG, Mol BW.
Randomised comparisons of medical tests: sometimes invalid, not always
efficient.
Lancet. 2000 Nov 25;356(9244):1844-7.
Some problems are:
1. a test may be perfect in diagnosing a condition (perfect accuracy), but
if there is no effective treatment, then the testing will not improve
outcomes despite its accuracy, as testing itself doesn't generally change
clinical outcomes, but it is the treatment that follows from it.
2. even if a test+treatment strategy is shown to improve clinical outcomes,
it does not necessarily mean the test has ANY value - the reason is best
illustrated with an example:
A large Cochrane review (with >30 RCTs) has shown aspirin reduces the risk
of pre-eclampsia in moderate or high risk women. These are simple
interventional trials comparing aspirin to placebo or no treatment in all
sorts of patient groups.
Now there are also some trials that have evaluated uterine artery Doppler (a
test to predict pre-eclampsia)+ aspirin treatment versus standard treatment.
If these trials show better outcome in the Doppler+asprin group, is that
proof that Doppler is a useful thing to do? No! Not necessarily. This is
because MORE women in the Doppler arm are likely to have received aspirin
compared to the no Doppler arm ANYWAY (even if the Doppler test had no
accuracy and indiscriminately labelled a proportion of women as "test
positive" & and if aspirin were to be generally effective in reducing
pre-eclampsia, then it is possible that Doppler arm would have shown benefit
regardless of whether Doppler test was a good predictor of pre-eclampsia or
not, or indeed, whether the Doppler test was done or not. So the better
outcome could be due to more receiving aspirin in the Doppler group than
there being anything special about Doppler!
I think if a test itself can modify the outcome (eg amnio or CVS can cause
miscarriage), then RCT has a role in their evaluation. I also think RCTs
have a role in screening test. In other circumstances, generally, i think
RCT has no role in the evaluation of a test - and the approach should be to
use a test to work out the probability of disease, and then estimate the
absolute effect that can be expected by treating with intereventions that
have been tested in RCTs, by following the concepts given in one of paul's
work:
Glasziou PP, Irwig LM.
An evidence based approach to individualising treatment.
BMJ. 1995 Nov 18;311(7016):1356-9.
Arri Coomarasamy,
Guy's Hospital
London
-----Original Message-----
From: Evidence based health (EBH)
[mailto:[log in to unmask]] On Behalf Of Paul Glasziou
Sent: 03 May 2006 22:14
To: [log in to unmask]
Subject: Re: Evidence for diagnostic tests from RCTs?
Dear Abdelhamid,
Can I expand a little on Bob Phillips point? The right study and
hierarchy depends on the questions you want to answer.
For diagnostic ACCURACY a cross sectional study with full verification
with a blinded adequate gold standard is fine.
That works even if we can't or don't want to treat, e.g, an untreatable
tumour or Osgood-Schlatter's disease.
But we may also want to know whether having the test available as part
of a complex management strategy improves outcomes. But that is a
complex brew of:
1. the test accuracy
2. the alternative tests
3. the available treatments
4. how clinicians interpret the test and behave as a result
So the RCT tests all this brew together (and probably more), not the
test accuracy. That is of greater interest, but usually less generalisable.
However, I agree that some tests need this, e.,g. cardiotocography in
labour. But others clearly don't - I am happy that the whispered voice
test* is accurate for diagnosing deafness without need an RCT to know
that it changes my or the patient's behaviour, or that the jolt
accentuation test can rule out meningitis (though better cross sectional
studies are needed to verify this).
Cheers
Paul Glasziou
* Pirozzo, et al BMJ 2003.
> Dear listers,
>
> I have been discussing with a colleague the best evidence
> about diagnostic tests. In our discussion, I found that some tutors
> take the oxford hierarchy of evidence at
> http://www.cebm.net/levels_of_evidence.asp
> at its face value while I see that RCTs can give also better evidence
> than cross-sectional studies if they are feasible to perform. One of
> the pitfalls of the hierarchy is that we may miss a better piece of
> evidence as it doesn't mention RCTs or systematic reviews of RCTs.
>
> Teaching EBM, I always concentrate on the concept of bias in different
> study designs and their implications on the process of evidence
> generation rather than restricting my students to a "rigid" scheme and
> I always ask them to start with searching on SR of RCTs, RCTs, etc..
> down the hierarchy of evidence.
>
> For a live example:
> In the Cochrane Database of SRs there is a systematic review for RCTs
> comparing the accuracy and safety of chorionic villous biopsy and
> amniocentesis in the diagnosis of genetic abnormalities.
>
http://www.mrw.interscience.wiley.com/cochrane/clsysrev/articles/CD003252/fr
ame.html
>
> This piece of evidence about diagnosis is of course better than 10s of
> cross-sectional studies. I think that we should add RCTs and their
> systematic reviews at the very top of the evidence pyramid for
> diagnosis as we do for therapy and harm?
>
> Any thoughts?
>
> Best of wishes,
> Abdelhamid Attia
--
Paul Glasziou
Director, Centre for Evidence-Based Medicine,
Department of Primary Health Care,
University of Oxford www.cebm.net
ph +44-1865-227055 fax +44-1865-227036
|
