On 26-Nov-04 ADRIAN ROBERTS wrote:
> Dear List,
>
> I recently ran one of a series of workshops for general
> practitioner registrars, on aspects of EBM. They were
> intelligent and receptive and it appeared to have gone well.
> However, I was intrigued by a conversation at lunch when
> one of the participants was earnestly and seriously
> recommending that her friend should try a particular herbal
> remedy for her ailment.
> The justification was, "It cleared my problem up really quickly."
> It demonstrated that the messages of that day and of previous
> sessions had not really got through. There was still the
> uncritical acceptance of anecdote as a basis for decision making,
> the failure to distinguish Necessary from Sufficient evidence,
> not to mention [...]
>
> This is not the first time I have felt that the teaching of EBM
> may not be reaching deeply enough and may not address the basic
> issues about what individuals accept as "evidence". [...]
> [...] Does my young doctor demonstrate the need for students
> to examine and understand their own underlying philosophy
> before EBM is "grafted" on? Will there not be "rejection" if
> the underlying personal framework is incompatible with EBM
> philosophy?
Leaving aside the issue of whether that particular herbal remedy
may or may not have pharmacological effect (as infusions of
willow bark or of Papaver somniferum certainly do), one can
recognise here a possible instance of what one might crudely
describe as belief in Magic (not necessarily called "magic";
less pejorative could be "folklore", which of course subsumes
"anecdote").
One could generally describe this as performing an action which
is not understood in the belief or in the hope that it will
produce a particular effect, the association between action
and effect having been established by passing on of the word.
As such, it forms part of (as you put it) the "personal
framework" of the practitioner. To the extent that this is
routinely applied in everyday situations, and not in extremis
[I can understand how someone for whom all else has failed
might try something weird "just in case", as a last resort]
it represents, bluntly, an inability to simply *think*.
Logic (call it epistemology if you wish, and it certainly
includes the distinction between Necessary and Sufficient
evidence) should be a part of thinking. All too often it is
absent when it ought to be present, and I hardly ever see
any reference to the teaching of Logic in most University
curricula, let alone in Secondary/High School curricula.
Just visit the UK Qualifications and Curriculum Authority
web page (this body is responsible for the content of UK
school curricula up to the age of 19):
http://www.qca.org.uk/
enter "logic" into the search box, and see what you come
up with. Out of 18 hits, the only references to "logic"
that have a remotely technical aspect arise in connection
with Mathematics and conputers (under IT). Even there it
can hardly be described as heavy.
You may think you've done a bit better if you search on
"thinking", which throws up hundreds of hits, many on
"critical thinking", and indeed one of these:
http://www.qca.org.uk/downloads/aea_critical_thinking.pdf
("DRAFT VERSION OF THE CRITICAL THINKING TEST SPECIFICATION")
doesn't look too bad on first reading until you start to
ask yourself "What *specifically* are they talking about
here?". And, in any case, it's dated Tue, 23 Nov 2004, a
mere 3 days ago, so we aren't there yet -- it's only a draft!
You can get an outline of assessment criteria for Critical
Thinking at
http://www.qca.org.uk/downloads/cos_pd(1).pdf
which is still at the rather vague and general level, leaving
a lot to the choice and judgement of implementers.
One might expect to find more under "philosophy" and even
expect to find something under "epistemology". There is only
one resource shown as available after a search on "philosophy",
which is "A level performance descriptions: philosophy", whose
text is identical to the "A level performance descriptions"
for other subjects. So there's no prescription for what one
should find in a course on Philosophy. (You can also find stuff
on "History, Philosophy and Theology" and "Philosophy and Ethics").
There are no matches for "epistemology" at all.
In short, proper education in thinking, including logic and
strict reasoning, does not seem to form part of the curriculum
in the UK. It may be different for some other countries: The
traditional French Baccalauréat is famous for making efforts to
do this sort of thing properly at pre-University level, and some
of its ideals have shaped the so-called International Baccalureat,
but from what I know of this as implemented in British schools
it has become somewhat softened in adoption by this country.
In the absence of rigour, people will "think by feel", and
I suspect, Adrian, that you have observed an instance of this.
> Others have commented on the increasing interest in
> complementary therapies, many with little empirical evidence
> to support them, among the new generation of British general
> practitioners.
I have heard soemthing of the beginnings of a movement in real
research departments to carry out properly controlled trials
of "complementary therapies", though little of interest seems
to have emerged so far.
One the other hand, one only has to browse through the "Lifestyle"
and "Alternatives" sections of the Sunday newspapers to see
the most bland and unsupported assertions of the efficacies
of all sorts of substances and creams in pots. The fact that
these are directly linked to the names of specific products
(with details of suppliers and prices) might nevertheless
appear to show that some research (sufficient to supply said
details) has been done; unless of course these details somehow
arrived out of the blue on the desks of the journalists.
In the absence of direct Evidence either way, I am not of
course going to assert any conclusion about this question ...
In particular, whatever suspicions I may hypothetically
harbour, it would be improper to assert that there is a rich
market in gullibility.
Gullibility, however, is at the root of belief in magic,
superstition, anecdote, folklore, call it what you will.
> Could it be more important for the educated practitioner
> to have a clear understanding of his/her own underlying
> frameworks, and where his/her decision making is coming
> from, than to understand a likelihood ratio for instance?
Quite frankly, anyone who can *understand* a likelihood
ratio should be capable of thinking clearly and logically.
This brings up yet another area of contention, where
gullibility also has a role: the use of statistical
methods. The logic of statistics is subtle and complex,
and much more bewildering than the logic of (say) mathematics.
What makes mathematics particularly hard is the technical
difficulty of the concepts and procedures, along with their
essentially abstract nature. To the extent that you can
succeed in driving the mathematical machinery, though, you
do know where you are and how you got there, and can rest on
a reassuring sense of certainty.
Statistics, however, is about the quantification of uncertainty.
At the end of a statistical analysis, you are still uncertain,
though you may be somewhat clearer as to what sort of thing
it is that you are uncertain about and, in that context, you
have a measure of how uncertain you should be.
In another distinction between statistics and mathematics,
one can say that the primary hypotheses from which mathematical
results follow are for practical purposes immutable, not in
question (leaving aside those remote frontiers where some
mathematicians are investigating whether these axioms are
appropriate or consistent). In statistical work, however,
though the tools are mathematical (and, once set in motion,
operate with mathematical determinacy), the hypotheses which
are adopted in practice are entirely at the whim of the
person doing the analysis. One should always be asking the
question "What other explanations of these results can I think
of?", over and above the tables of analyses of variance,
of effects and interactions and their P-values, etc., which
flow from adopting particular hypotheses ("models") for the
data. Possible examples of completely false conclusions,
resulting from failure to perceive alternative explanations,
are too numerous to mention here.
In particular the concept of "likelihood", properly understood,
requires you to be explicit about all the hypothese you are
contemplating as possible explanations. To the extent that
you cannot explicitly state the entire range of hypothesis
underlying a "likelihood ratio", you neither understand
"likelihood" nor how your statistical analysis works.
I can appositely quote from a message this morning to the
"r-help" mailing list (R being statistical software
closely related to S-Plus®):
"As I started out using SPSS when there was no
GUI (in fact, no interactive interface at all),
I automatically open up the syntax editing window
when I have to use it. It's a workable text editor,
you can run all or part of the code at will, and
build up a code file in much the same way as R.
On the other hand, it does encourage the user who
has not taken Pope to heart ("A little learning...")
to put their data through a high-powered analysis
while convincing themselves that they know what
they are doing. I confess to having done it more
than once in the past.
It was when I began reviewing other researcher's
papers, and thinking 'This guy didn't know what
he was doing.' and then, 'And you've done it too,
brother.' that I resolved to be more circumspect."
Yet how many courses -- even at University level -- on
"Research Methods", "Statistics and epidemiology", etc.,
pay proper attention to this? In how many, on the other
hand, are people inculcated into the technicalities of
running analyses on statndard software, choosing from
among the options presented by the drop-down methods,
and by default, in the end, *believing* their results
(and not even keeping in reserve the caveat implicit
in a P-value of 0.01, say, which implies that of the
10,000 people doing research on something, 100 will
get a false "significant" result and, God-Dammit,
getting them published).
Yes, I am aware that warnings of "publication bias" are
given; but I also think that many people don't really
take these seriously. They really do think that their
result is true, forgetting that statistics is intended
to deal systematically with *uncertainty*.
To the extent that people believe that uncertain conclusions
are true, this is Gullibility again.
Circumspection is, parhaps, an initial antidote to Gullibility!
Best wishes to all,
Ted.
--------------------------------------------------------------------
E-Mail: (Ted Harding) <[log in to unmask]>
Fax-to-email: +44 (0)870 094 0861 [NB: New number!]
Date: 26-Nov-04 Time: 13:17:53
------------------------------ XFMail ------------------------------
|
