I have followed the discussion on epistemology and logic with interest,
because it centres on an issue that I have been wrestling with for some
time. All systems for assigning a quality score, or grade/level of
evidence are difficult to apply in practice. For example, treatments
with effects so dramatic that a fair trial does not need a double blind
RCT (see http://www.jameslindlibrary.org/context/dramatic_effects.html)
It seems to me that there is a fundamental flaw with all systems for
scoring quality, or grading levels of evidence. These essentially try to
quantify how plausible a result or recommendation is. However, the road
to medical knowledge may be easier to follow if we try to quantify (or
at least assess in a transparent and systematic way) how sceptical we
should be about a claim.
A scepticism score is analogous to Karl Popper's analysis of what works
well and what works poorly in scientific discovery: setting out to prove
that a hypothesis is true is not as fruitful as trying to falsify the
hypothesis.
I am developing a Scepticism Score-Implausibility index for use in our
in-house critical appraisal methodology. If anyone is interested in
helping to develop the SS-II to a level at which it could be published
or validated, please contact me at [log in to unmask] I think that
an SS-II could cope with extreme cases such as plausible evidence for
dramatic treatments coming from small observational studies
(http://www.jameslindlibrary.org/context/dramatic_effects.html ), and
doubtful evidence coming from high quality meta-analysis of high quality
RCTs (e.g. Lancet 1997,; 350: 834-843).
Michael
Dr. Michael Power
Clinical Author
Sowerby Centre for Health Informatics at Newcastle Ltd
University of Newcastle
www.schin.ncl.ac.uk
[log in to unmask]
tel +44 (0) 191 243 6105
fax +44 (0) 191 243 6101
addr Bede House, All Saints Business Park, Newcastle upon Tyne NE1
2ES
Disclaimer
If you received this in error, please contact the sender and delete the
material from any computer. The information transmitted reflects the
thoughts and opinions of the author and does not represent the thoughts
or opinions of SCHIN Ltd and is intended only for the person or entity
to which it is addressed and may contain confidential and/or privileged
material. Any review, retransmission, dissemination or other use of, or
taking of any action in reliance upon, this information by persons or
entities other than the intended recipient is prohibited.
|