Cases for Comment have now been running for exactly 15 months. The first
Case for Comment attracted 24 participants; and I am delighted to say that
15 of these Founder members are still regular participants. The last year
has seen a continuing rise in the number of participants - there now have
been around 300 participants (including one regular 'group response' and an
electronic participant) from some 27 different countries. Cases now
frequently attract more than 30 individual participants each week.

The last 25 Cases have seen a major break-through - the formal evaluation
and scoring of comments on Cases. This has been based on breaking down
comments into separate components (many of which are common to many
participants); and using a group of assessors to score independently these
components 'blind' (ie without knowing the participant or the outcome of an
individual Case). The component scores have been distributed with the Case
Summary. The sum of component scores for has been recorded for each comment
from registered CME/CPD participant has been recorded, and we will send
these together with a ranking summary to individual participants in the
next few days.

I am very grateful to the assessors and the 9 additional participants who
last Summer helped in a validation exercise of this method of scoring
comments - the results of this will be published in Annals of Clinical
Biochemistry, I believe in the March 1999 issue. The method appears robust,
but has attracted some comment and criticism.
First, it has so far been true that 'phone the clinician' has attracted a
positive score. I think that this is the ideal - it is almost always
helpful to discuss a case with the responsible clinician. Unfortunately for
many of us the responsible clinician is frequently difficult to find; and
there is simply not enough time to discuss every problem case with a
clinician.
Second, the 'correct response' in terms of the eventual outcome of the case
has not always resulted in the highest scores. I think that this is
realistic: the assessors are marking on the basis of what seems reasonable
at the time a set of results is seen. For a case with an unusual outcome,
the 'more probable outcome' may score higher than the actual outcome.
Third, just as there have been major disagreements between participants in
their comments and further action, there have also been occasional major
disagreements between assessors in their component scores. There are
several reasons for this (see the Annals paper): participants have to
remember that we are all individuals in our own environment, and what may
seem highly appropriate to one may not be so to another. For the assessors,
such disagreements have not usually affected 'key features' in the
interpretation of a case: they have been around the appropriateness of
suggesting what may be an unusual cause for a set of abnormalities, or the
wisdom of suggesting further analyses or treatment. Interestingly, the only
apparent difference in scoring between assessors from a medical compared to
a scientific background is that assessors from a medical background tend to
give more extreme marks (ranging from -2 to +2) to suggested clinical
advice than those from a scientific background (scores ranging from -1 to
+1). This may of couse reflect personality rather than cultural
differences!

Personally I believe that giving component scores is of considerable value,
since it gives guidance as to the most important ideas which should be
included in a comment. I know I have modified some of my commenting
patterns as a result of experience with these scored Cases, and I hope
others will have done the same.
We will be carrying out a detailed study of all the comment scores over the
last 21 Cases, the results of which will hopefully be published later this
year. It is apparent that those who make long and discursive comments
containing many individual components tend to score slightly higher than
those who restrict their comments to two lines containing an average of
around three components (although one participant in a previous Case
managed to squeeze in 6 components into two lines through using
abbreviations, and scored very highly as a consequence). We will be looking
in detail at this aspect of the scoring, perhaps separating 'comment' and
'action', or perhaps giving a mean component score as well as a total
comment score. An important point for the future is that we ask
participants to respond as they would in reality and in real time, rather
than as might be suggested by detailed reading of the textbooks. There just
isn't enough time in real life for the latter (and it makes our job of
scoring comments and their components very difficult!).  

In the meantime, we would like to thank everyone who has participated in
these Cases, through comments, through suggestions and sometimes
criticisms, with further case suggestions, and with support for the idea.
Most of all we would like to thank the assessors, ''7 wise men and women'
who have contributed so much to this exercise.

With best wishes
Li Ping
Gordon Challand




%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%