Dear all
The message written by Andrew Booth raises some very pertinent points.
> However would it not be the case that it is not worth doing a systematic =
> review of a therapy unless there is already a very reliable method of =
> diagnosis for the condition?
Some of the problems with studies that assess the performance of a diagnostic test
(i am writing from a radiological perspective) is that:
1. A rigororus methodological design (CBA, RCT etc) is often not applied (for various
resasons however, e.g. rapidly evolving technology and diffusion);
2. There is a whole array of biases that can be introduced with regards to the general
design of the study e.g. representative sample, valid and appropriately applied
reference ("gold") standard, observer variability etc. [1]; and
3. Incorrect use and interpretation of indexes of accuracy (Sn, Sp) and analysis in general.
On occasions, it may well be debatable as to whether there is a very reliable
diagnosis from which to base clinical management on, and an even more
tenuous link as to how the diagnostic technique has an effect on patient outcome.
The evaluative framework that was first used to address this problem was suggested
by Fineberg et al (1977) [2] and has since evolved into a 5-level hierarchial framework [3, 4].
The 5 categories are: Technical perfromance; diagnostic performance;
Diagnostic impact; Therapeutic impact and Impact on health. However,
the message written by A Booth implies that systematic reviews are often conducted
in relation to diagnsosis and therapy as separate levels of the global framework.
Perhaps more systematic reviews should have a broader scope, rather than
only concentrating on diagnosis, therapy, prognosis etc, as single levels of the
above framework (although this assumes that at least some studies address the
research question at all these levels as well).
>However to take a review that I am currently undertaking namely =
>"acupuncture in migraine" I was speculating whether the review is =
>potentially strengthened by the existence of the International Headache =
>Society's criteria for diagnosing migraine, or maybe this is irrelevant?
I think this would be useful as part of the process of developing your eligibility
criteria i.e. only include studies that used the Society's criteria for
diagnosing migraine - i don't know whether there is a reliable way of "definitively"
diagnosing a patient with a migraine.
The reliable method of diagnosing a patient is important however, if you use MRI/CT
for example, to diagnose condition X, as the way in which the technology was initially
evaluated affects the reliability of the diagnosis that you can make. Therefore, if you
were conducting a review of a therapy on the basis of a diagnosis that was made by either
MRI or CT i.e. some patients were diagnosed by MRI then received therapy, some patients
diagnosed by CT then received therapy you would want to enure that the two diagnostic
technologies had both been rigoroulsy evaluated and found to be equally efficacious and
effective, although in this scenario it would probably be more cost-effective to use CT so only
CT would be used in normal practice rather than MRI. Having said this therefore, it
is probably more important to ensure that the patient is correctly diagnosed before you start
assessing the effectiveness of a particular therapy, as the framework suggests....hmmm.
Regards,
Stephen Brealey
1. Kelly S et al (1997). BJR, 70, 1028-1035.
2. Fineberg HV et al (1977). JAMA, 238, 224-7.
3. Mackenzie R, Dixon AK (1995). Clin Radiol, 50, 513-8.
4. Begg CB (1987). Stats Med, 6, 411-23.
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
|