I have started to gather together some hints and tips on handling
critical appraisal sessions (heavily indepted to the Sackett et al
EBM book. I would be grateful for any additional suggestions that
might be added in this particular format. I shall compile any
suggestions into a Web page that I will add to my Netting the
Evidence guide. Your suggestions would be most appreciated.
What do I do if no-one has read the article beforehand?
Allow some reading time. However it is seldom productive to have them
read the whole article thoroughly, thus consuming valuable discussion
time. Get them to read the edited highlights - e.g. the structured
abstract, results and conclusion plus the baseline data table and the
results tables/diagrams. If it is a BMJ article also get them to read
the key points box. Spend no more than five minutes on reading
through.
What do I do if most people have read the article but a couple haven't?
Rather than have the non-prepared group members
reading while the rest are working you could have one of the prepared
members describe the study and its findings to the group as a whole.
You would have to make sure that they merely described the factual
content of the study and that they didn't seek to impose an
interpretation on it.
What do I do if one person hogs the discussion?
Use a summary technique that includes every one else but excludes that
person by implication e.g. "Do the rest of you agree with that point?"
"Does anyone else have a different perspective on that issue?"
What do I do if one person is not voicing an opinion?
Gently invite their
opinion, usually on a simple or straightforward topic preferably in
their specific domain. Alternatively you could vote on an issue and
then invite ones who voted either for or against to explain their
decision and include the non-participant in those you select. Above
all do not pressure someone into involvement - once you have tried
unsuccessfully to include someone cut your losses. Nothing is gained
by trying repeatedly to include someone against their will - you will
just interfere with the group dynamic and embarass yourself and the
non-participant. An exception is where the group is likely to be in
existence for some time (e.g. a journal club) where you could rotate
responsibilities to present or summarise.
What do I do if there is a deathly silence from the whole group?
David Sackett's advice is to
"wait the magic 17 seconds". Apparently no group can abide silence
for this long and as long as the facilitator doesn't give in first
your patience will be rewarded.
What if I get asked a question that I can't answer?
Use the group as a resource. Ask "Would anyone else like
to explain this?". Many people like the opportunity to show what they
know and, by explaining it to others it reinforces learning for the
individual and for the group as a whole. If the group is asking a
large number of questions to which neither you nor they know the
answer then take time away from the article to list the group's
learning needs and to delegate members of the group to go away and
research the answers.
What if I get asked a question that no-one can answer?
Make a point of formally recording the outstanding question on
a flip chart or whiteboard. At the end of the session delegate someone
from the group to find the answer for next time. At the beginning of
the next session handle this report-back first to achieve a sense of
closure. Above all attempt to distinguish those questions that will
interfere with interpretation and appraisal of an article (NEED TO
KNOWS) from those that the group would just like to answer (LIKE TO
KNOWS).
What if it is a tricky statistical point?
There is no need to
necessarily handle this any different from any other learning need.
Again if it severely compromises the understanding of the paper then
the answer will have to be found before proceeding with the article.
However if it is just a question of an unfamiliar method or technique
then ask the group to take that on trust for the moment and to check
this later after the session - point out that the statistics is one of
the easiest sections of the paper to validate, (compare inadequate
description of randomisation, for example), and that if an
inappropriate method has been used it will often be picked up by
statistical reviewers. Two useful resources to carry with you are
Last's Dictionary of Epidemiology and Greenhalgh's How to read a paper
(especially p.73).
What if a group is approaching "critical appraisal
nihilism"? This expression, which refers to the tendency to "trash
papers", was coined by David Sackett. Distinguish between major flaws
and minor flaws. Ask the group to respond to criticisms by stating
how they would do this better - in many cases they will come to
recognise that practically it would be very difficult to improve on a
study. e.g. blinding patients to a particular distinctive package of
care. Above all, stress the three major flaws that have a bearing on
reliability; defects in randomisation, blinding and failure to account
for withdrawals or drop-outs.
Further reading: Sackett DL. et al. Evidence-based Medicine: How to
practice & Teach EBM. London: Churchill Livingstone, 1997. pp. 188-199
Andrew Booth BA MSc Dip Lib ALA
Director of Information Resources
School of Health & Related Research (ScHARR)
Regent Court
30 Regent Street
SHEFFIELD
S1 4DA
Tel: 0114 222 5420 or 5214 Fax: 0114 272 4095
The author of Netting the Evidence:
http://www.shef.ac.uk/~scharr/ir/netting.html
and Trawling the Net:
http://www.shef.ac.uk/~scharr/ir/trawling.html
E-mail: [log in to unmask]
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
|