In an earlier age I would have been burned at the stake as a heretic, so I
am glad to be living in the Internet age!
I think that the picture of the 5S hierarchy as a pyramid is misleading
for three reasons.
First: Evidence from a synopsis of an individual study should be below
evidence from the level of a systematic review. And, evidence from a
synopsis of a systematic review should be (just) above evidence from a
systematic review.
Second: A better representation of the structure of evidence would be a
tree (upside down if you want to give the impression of getting narrower
towards the "top"). The structure could be represented by a pyramid, but
then the graphic should show the individual stones in each layer. Each
layer should also show several gaps between groups of stones to give the
impression that evidence can be collected into bodies of evidence on
different issues. (The best metaphor that I have come across for the
structure of evidence is an infinite crossword puzzle. One square
corresponds approximately to what we understand as "atomic" evidence, e.g.
one outcome measure in one RCT. A "word" in the crossword puzzle
corresponds to a body of evidence: the answer is validated by checking,
not only the clue, but also the clues and answers of all related words in
the puzzle.)
Third. "Evidence" is a slippery, fuzzy, polymorphous concept that
invisibly/subconsciously changes shape at each layer of the pyramid.
Because "evidence" invisibly changes its meaning at each layer this
conceals the different nature of each layer. If you think of each layer of
the hierarchy as providing answers (i.e. evidence) to different types of
questions, you can see that the optimal search strategies are likely to be
different for each layer of evidence.
At the bottom layer of the pyramid, the questions are very specific and
require "atomic" types of evidence, for example "is intervention I in
population P more effective than placebo C as measured by outcome O at
time T".
Towards the top of the pyramid, questions become more general. For
example, to answer "How do I manage someone with condition X?" you need to
consider all the options for management, and to weigh up all the pros and
cons of all the options. Evidence at this level consistes of several sets
(or bodies) of evidence on benefits, adverse effects, tolerability,
economics, availability, etc.
I think the the 5S pyramid represents a ranking of information resources,
not a hierarchy of evidence.
The layers "map" to a set of questions that the information resources can
answer. The higher the level, the more general the question you can find
answers to, and the more complex the body of evidence required to provide
the answers.
The layers also represent and rank the values added by the different types
of publication resource. You can measure value in terms the effort you do
NOT have to put into searching, appraising, and synthesizing evidence
because someone has done the work for you. The higher up the pyramid, the
more work has been done for you.
Michael, ducking his head below the parapet
|