Print

Print


Nicola Innes writes:

>I'm sure I remember reading somewhere that in RCTs there can be a
phenomenon
>whereby the control outcome is often better than that expected from the
pool
>sampled from. Can anyone give me more details on this please?

This could be referring to several things.

The first issue relates to the Intention to Treat approach. If you exclude
patients who are not compliant, you will have problems. There is empirical
data which shows that patients who fail to comply with a placebo have worse
outcomes than patients who do comply with placebo. While it is tempting to
exclude non-compliant patients, it harms generalizability. What doctor has
the option of only treating compliant patients?

Second, sometimes the inclusion/exclusion criteria will create a healthier
pool of patients than the general population. You might exclude patients
with co-morbid conditions, or above a certain age. Even a simple restriction
on language might be problematic. Patients in the United States who do not
speak English well are likely to have a lower socio-economic status (SES),
and SES is associated with prognosis for a variety of diseases.

Third, sometimes the unhealthiest patients are the first to drop out of a
study. Treatment failures may decide to leave the country to start Laetrile.
Also, highly mobile people in most societies tend to be low SES (they have
to go where the jobs are).

Fourth, the standard of care in a control group is usually better than that
of the general population. They get seen more regularly by medical
professionals and they may have extra reminders or incentives to comply with
all aspects of their therapy.

Generally, but not always, all of the factors that cause the control group
to have better outcomes in a RCT also influence the treatment group equally.
So the internal validity of the study is not an issue. The problem is
external validity, or the ability to generalize beyond the study population.

I'd be interested in citations along these lines. For example, there was a
report that got a lot of press a couple of years ago that most AIDS trials
would greatly overstate the effectiveness of the treatments being studied.
Does anyone know where that got published?

Steve Simon, [log in to unmask], Standard Disclaimer.
STATS: STeve's Attempt to Teach Statistics. http://www.cmh.edu/stats
Watch for a change in servers. On or around June 2001, this page will
move to http://www.childrens-mercy.org/stats