Thanks, Michael, thoughtful stuff
>> I think the 5S pyramid represents a ranking of information resources,
not a hierarchy of evidence.
Absolutely. IMHO this is fundamental to understanding its role. It
answers the question "in what order should I search to find the answer
to my question?" It doesn't answer the question "what is the best level
of evidence?" - that question is answered by those producing the top 4
layers. Eg if the best available evidence is observational studies, or
poor quality RCTs, or whatever, a good evidence-based textbook will say
that and also indicate the possible limitations and caveats that follow
Slawson and Shaughnessy (J Am Board Fam Pract 1999; 12: 444-9) talk
about the usefulness equation. Usefulness = relevance * validity
*(1/work required)
We all know about validity (eg checking for bias, statistical analysis,
suitable comparators, etc, etc). Relevance is described (for most
front-line practitioners) by FOCC - is the intervention Feasible in my
setting, is the Outcome of importance to patients (eg CHD death or
non-fatal MI vs LDL lowering), and, when foraging ie keeping abreast of
current developments, cf hunting for the answer to a specific question,
is the condition Common in my practice and would the information, if
true, require me to Change my practice
But most front-line practitioners would have to do a lot of work to
obtain and assess the validity of original papers and put the
information into the context of the rest of the evidence base. They
have little time and may not have the skills to do this, so the
usefulness is likely to be low. The higher up the 5S pyramid we go, the
less work is required, and the greater the validity and (probably)
relevance of information (as invalid and irrelevant information has been
screened out) - thus the usefulness is enhanced.
IMHO we must distinguish between the type of skills required by front
line practitioners to enable them to practice EBM, and the lit
searching, critical appraisal etc skills required by those producing the
top four of the 5s. Otherwise front-line folks think they need to be
whizzo critical appraisers, become disheartened and revert to brief
reading, asking "experts", experience, and "I think"
Cheers
Andy
Andy Hutchinson
email: [log in to unmask]
tel: 07824 604962
web: www.npc.co.uk or www.npci.org.uk
-----Original Message-----
From: Evidence based health (EBH)
[mailto:[log in to unmask]] On Behalf Of Michael Power
Sent: 21 February 2008 14:00
To: [log in to unmask]
Subject: Re: The first step to search evidence - use the "4S" structure
(Haynes, 2001)
In an earlier age I would have been burned at the stake as a heretic, so
I
am glad to be living in the Internet age!
I think that the picture of the 5S hierarchy as a pyramid is misleading
for three reasons.
First: Evidence from a synopsis of an individual study should be below
evidence from the level of a systematic review. And, evidence from a
synopsis of a systematic review should be (just) above evidence from a
systematic review.
Second: A better representation of the structure of evidence would be a
tree (upside down if you want to give the impression of getting narrower
towards the "top"). The structure could be represented by a pyramid, but
then the graphic should show the individual stones in each layer. Each
layer should also show several gaps between groups of stones to give the
impression that evidence can be collected into bodies of evidence on
different issues. (The best metaphor that I have come across for the
structure of evidence is an infinite crossword puzzle. One square
corresponds approximately to what we understand as "atomic" evidence,
e.g.
one outcome measure in one RCT. A "word" in the crossword puzzle
corresponds to a body of evidence: the answer is validated by checking,
not only the clue, but also the clues and answers of all related words
in
the puzzle.)
Third. "Evidence" is a slippery, fuzzy, polymorphous concept that
invisibly/subconsciously changes shape at each layer of the pyramid.
Because "evidence" invisibly changes its meaning at each layer this
conceals the different nature of each layer. If you think of each layer
of
the hierarchy as providing answers (i.e. evidence) to different types of
questions, you can see that the optimal search strategies are likely to
be
different for each layer of evidence.
At the bottom layer of the pyramid, the questions are very specific and
require "atomic" types of evidence, for example "is intervention I in
population P more effective than placebo C as measured by outcome O at
time T".
Towards the top of the pyramid, questions become more general. For
example, to answer "How do I manage someone with condition X?" you need
to
consider all the options for management, and to weigh up all the pros
and
cons of all the options. Evidence at this level consistes of several
sets
(or bodies) of evidence on benefits, adverse effects, tolerability,
economics, availability, etc.
I think the the 5S pyramid represents a ranking of information
resources,
not a hierarchy of evidence.
The layers "map" to a set of questions that the information resources
can
answer. The higher the level, the more general the question you can find
answers to, and the more complex the body of evidence required to
provide
the answers.
The layers also represent and rank the values added by the different
types
of publication resource. You can measure value in terms the effort you
do
NOT have to put into searching, appraising, and synthesizing evidence
because someone has done the work for you. The higher up the pyramid,
the
more work has been done for you.
Michael, ducking his head below the parapet
|