In a private e-mail, one of the participants of this list suggested that my
"Content and quality of 2000 controlled trials in schizophrenia over 50
years" Ben Thornley and Clive Adams BMJ 1998; 317: 1181-1184.
devalued the work of people in this area and that words like "terrible" and
"mess" were needlessly cruel. I'm sorry that my characterization of this
research paper was so extreme. A more accurate description can be obtained
by quoting from the authors themselves.
"The quality of reporting was poor. Only 4% (80) of the trials clearly
described the methods of allocation. Explicit descriptions of blinding were
adequate in only 22% (440) of trials, while some description of treatment
withdrawals was given in 42% (840)." page 1182.
"Only 20 trials (1%) raised the issue of the statistical power of the study.
The average size of schizophrenia trials was small. For an outcome such as
clinically important improvement in mental state to show a 20% difference
between groups a study would have to have 150 participants in each arm
(alpha=0.05, power 85%). Only 3% (60) of studies were of this size or
greater. More than 50% of trials had 50 or fewer participants." page 1182.
"In all, 510 (25%) studies did not use rating sclaes to measure outcomes.
The remaining 1490 trials used 640 different instruments." and later
"Overall, 369 scales were used only once." page 1182.
"The quality of reporting in this large sample of trials was poor and showed
no sign of improvement over time. As low quality scores are associated with
an increased estimate of benefit, schizophrenia trials may well have
consistently overestimated the effects of experimental interventions." page
"Further difficulties with using the evidence generated by this mass of
research are that the studies are of limited duration for an illness that
often lasts decades." page 1183.
"The findings of this survey are as bad, if not worse, as those for other
disciplines of health care. Certainly, there is a long way to go before all
interventions for patients with schizophrenia have been adequately evaluated
and systematically reviewed and some of the enduring questions about the
efficacy of treatment are answered." page 1183.
I hope that the direct quotes I have provided give a better sense of these
issues, which, in my opinion, are important for anyone who practices
Evidence Based Medicine.
Steve Simon, [log in to unmask], Standard Disclaimer.
STATS - Steve's Attempt to Teach Statistics: http://www.cmh.edu/stats