|
Thank you to the following for their helpful responses to my query of 11th
Jan: Doug Altman, Martin Bland, Eddie Channon, Margaret Eames, Alex M.
Gray, Richard Gray, Alex McMahon, Phil McShane, Jeremy Miles, Robert
Newcombe, Sima Patel, Carrol Preston, Paul Seed, Bob Shaw, Andy Vail, Jay
Warner and Mike Williams. These are summarised and listed below.
My original query was as follows:
"I'm advising a colleague on their protocol for a study, and I would like
some advice on whether my approach is reasonable or flawed, since it
doesn't strictly conform with Altman's approach.
The study involves a potentially life-threatening condition, for which
treatment is routinely commenced prior to laboratory confirmation of the
diagnosis. Upon obtaining a non-confirmatory result alternative diagnoses
and treatments are then used, as appropriate.
Ideally patients would not be randomised into the planned study until the
test has been done and the diagnosis confirmed (using clearly defined
criteria), but for various reasons this is not likely to be possible in a
large proportion of patients who consent and are eligible on the other
criteria. Those patients in whom the diagnosis is not confirmed should
then be removed from the study when these lab results are known, since they
do not belong to the intended target population for the study.
I propose that all data for these ineligible patients should then be
excluded from the analyses, since they do not belong to the target
population. Altman (1991, p 464) argues that all patients randomised
should be included, in an intention to treat analysis, and that this must
be considered the main analysis, to avoid subjective decisions providing an
opportunity for bias.
In the study we're considering all patients would experience the same
laboratory testing (sample taken prior to initial dosing), and an objective
rule would be used to determine which patients do not in fact meet all the
inclusion criteria, so no opportunity for bias would arise.
Is there a flaw in my approach?"
SUMMARY OF RESPONSES
======================
(This summary freely quotes from responses. They are each listed below
this summary, with names removed.)
Five respondents preferred my proposed approach of focusing on the Per
Protocol or modified ITT population, while the majority advocated using
both the strict ITT and the modified one.
Use of the modified ITT approach is subject to the following provisos:
1 The criteria for exclusion of patients should be clearly defined and
objective;
2 They should be pre-specified in the protocol;
3 The decision to exclude or otherwise should be made blind to the
allocated treatment;
4 Diagnosis should be made on data not relating to events after
randomisation;
5 The treatment may be harmful in patients who do not have the disease. So
it is important to report safety for all patients treated.
We anticipated that all of these would apply in the study (which I misle
adingly referred to as a condition) concerned. In view of the need to
initiate other possible therapies in the event of non-confirmation of
diagnosis the exclusion would be made as soon as the lab results are
available.
In support of the modified ITT approach it was noted that including
patients in whom the drug cannot work might lead to qualitative
interactions with treatment. But this may miss the fact that the treatment
has an undesired effect on some misdiagnosed patients. The effects may be
acceptable in those with the life-threatening condition, but not in those
with something more benign that have been mis-diagnosed. For this reason
both approaches are advocated. Neither analysis would be regarded as more
appropriate than the other - they're simply addressing two questions, which
though closely related, are importantly different.
A constraint to ITT arises in loss to follow up of patients. This is where
the difference in the two approaches is perhaps most apparent. If the ITT
approach is to be used then there is need to follow-up patients who are
excluded, to ascertain their outcome, for safety purposes.
Studies in the following areas were mentioned as ones in which the modified
ITT approach has previously been used: antibiotics, Cholera, Parkinson's
Disease and Vitamin supplements in pregnancy. The study I was asking about
concerns meningitis.
Please note that I did not intend to attribute the ITT approach solely to
Altman, but his was the only text available to me at the time here in
Malawi. I have since also seen another text on Randomised Clinical Trials.
Thanks again to all respondents
Sarah A White
Statistician
Malawia-Liverpool-Wellcome Clinical Research Programme
PO Box 30096, Chichiri
BLANTYRE 3, MALAWI
RESPONSES RECEIVED
===================
(listed in the sequence received)
RESPONSE 1
It is fine to exclude them if there are clearly defined objective criteria
to do so. However, there are usually 'grey area' cases and so it is best
for the laboratory assessment and eligibility decision to be made blind to
allocated treatment. Once eligibility has been determined, then the
eligible patients are your intention to treat population and all these
patients should be included in the outcome analyses even if doubts arise
later about diagnosis. Similarly ineligibility is irrevocable even if shown
later to be mistaken.
RESPONSE 2
After a quick read of your email it could be that your study is one of the
occasions when an active treatment run-in period is useful. Although I
suspect that the active treatment has to be prescribed as soon as there is
any suspicion of a patient having the disease (in which case the run-in
period is no use). Off the top of my head I would not be keen on seeing
this as a so-called 'pragmatic' study, where the patients are considered to
be randomised to 'treatment strategies'. Having the disease is important
because including patients in whom the drug cannot work may lead to
qualitative interactions with treatment.
Intention to treat isn't *always* applicable, although it virtually always
is. Here is an excerpt from one of my papers (Statistics in Medicine in
press)...
" Although the intention-to-treat analysis is the analysis of choice [1],
it is possible to imagine unusual circumstances when the study may have to
do without it. Randomisation may have taken place long before the start of
treatment (this might well be necessary in rare occasions), so that
researchers may have to argue that the intention-to-treat analysis is not
strictly applicable. This situation can sometimes be remedied by simply
carrying out the randomisation as late as possible. Hollis and Campbell
provide an example of a surgeon who achieved randomisation by tossing a
coin in the operating theatre after the patient had already been cut [2]. "
1. Chene G, Morlat P, Leport C, Hafner R, Dequae L, Charreau I, Aboulker
JP, Luft B, Aubertin J, Vilde JL, Salamon R. Intention-to-treat vs on
treatment analyses of clinical trial data: experience from a study of
pyrimethamine in the primary prophylaxis of toxoplasmosis in HIV-infected
patients. Controlled Clinical Trials 1998; 19: 233-248.
2. Hollis S, Campbell F. What is meant by intention to treat analysis?
Survey of published randomised controlled trials. BMJ 1999; 319: 670-674.
RESPONSE 3
No, this is a case in which clearly the general preference for ITT needs to
be relaxed. In this situation you would perform and report two sets of
analyses - ITT analyses including all patients who were randomised, and
restricted analyses including just those for whom the diagnosis was
confirmed. Neither analysis would be regarded as more appropriate than the
other - they're simply addressing two questions which though closely
related, are importantly different. The CONSORT guidelines for reporting
RCTs strongly emphasise the need for a flowchart to show what happens to
each subject who is enrolled in (or even considered for entry to) an RCT.
This principle applies particularly strongly here. You would of course
want to check that the proportions whose diagnosis was changed was
(broadly) similar in the two groups - though hypothesis testing isn't
relevant here as any difference should merely be a consequence of random
allocation.
RESPONSE 4
Your approach is not flawed, but Altman is also correct - the primary
analysis should be an intent-to-treat basis which includes all subjects
that take at least one dose of study medication. The population you are
describing is what we would call a Per Protocol population, and (at least
in the work I do) we would always state this population would be identified
prior to unblinding the randomisation and the primary measure of efficacy
analysed for both this population and the ITT population. Provided (a) it
is stated in the protocol that this will be done, and (b) all exclusions
are identified before unblinding, then your approach is fine.
RESPONSE 5
I am sure Altman would agree that his approach might sometimes need
modifying.
However, apart from the problem of bias there is another difficulty: the
treatment may be harmful in patients who do not have the disease. This may
be sufficient to outweigh any benefit in those who do. The only way of
testing this is an 'intention-to-treat' analysis. If this were the case
then it would be necessary to alter the policy of commencing treatment
before diagnosis.
I would be inclined to do both analyses (I assume there is no problem in
this) and be quite explicit about it and discuss the reasons for the
difference if any between the results.
RESPONSE 6
I would agree with Altman's ITT approach for primary analysis. This
answers the clinically relevant question as to what clinicians should do
when a patient presents. The ITT approach has the more appropriate "target
population" of presenting patients. It may, for example, turn out that the
initiated treatment is good for people who actually have the disease, but
bad for those with alternative diagnoses. In such a circumstance, the
post-randomisation exclusion policy would be particularly dangerous.
I would however support a secondary analysis allowing your exclusions
provided the interpretation was very careful. The conclusions are not of
immediate clinical relevance, but may suggest that the treatment could in
future be useful if quicker diagnosis became available.
RESPONSE 7
What you are suggested is often termed as a "modified ITT", we use it
routinely in the investigation of the efficacy of antibiotics. Here too
treatment is started before we know whether the patients signs and symptoms
are caused by a pathogen (for which the treatment will be effective) or
not.
You haven't said anything about the disease or the trial design. In our
case, when the decision is made to exclude the patient from the ITT we are
still blind to the treatment and outcome, therefore free of potential
selection bias. If you can have a blinded and preferable independent
review of the lab results to determine whether they have the disease and
therefore are Mod-ITT that would strengthen the validity of your approach.
Irrespective of the ITT issue, it is still important that you report safety
(adverse events, safety labs etc.) for all patients treated.
RESPONSE 8
We had something very similar in our study of vitamin supplementation in
pregnancy to prevent pre-eclampsia. (Chappell LC, Seed PT, Briley AL,
Kelly FJ, Lee R, Hunt BJ,Parmar K, Bewley SJ, Shennan AH, Steer PJ, Poston
L. Prevention of pre-eclampsia by antioxidants: a randomised trial of
Vitamins E and C in women at increased risk of pre-eclampsia)
The main entry criterion was 2 abnormal doppler ultrasound tests; but women
started on vitamins after the first. I analysed it both ways (ITT &
per-protocol), and we wrote it up with the emphasis on the ITT analysis.
There are two problems with only using per-protocol:
* You may unbalance the groups if the treatment affects the diagnosis
* You may miss the fact that the treatment has an undesired effect on some
misdiagnosed patients. The effects may be acceptable in those with the
life-threatening condition, but not in those with something more benign
that have been misdiagnosed.
ITT is more-or-less the only acceptable option for the final phase of
pharmaceutical research ("Phase III" studies). The CONSORT statement
(http://www.consort-statement.org/) & the ICH guidelines
(http://www.ifpma.org/ich1.html), particularly guideline E9
(http://www.ifpma.org/pdfifpma/e9.pdf), leave no doubt about this.
RESPONSE 9
Intention to treat is a concept I have great difficulty understanding,
because many people define it such different ways. I have now kind of
accepted that true ITT is as defined by Altman. In your problem I see what
you mean about the fact that subjective decisions about the patients is
highly unlikely. However I think I would still go with Altman's explanation
because given good randomisation and allocation concealment, you should
theoretically have equal numbers of negative patients in both groups, and
hence a fair and informative comparison of the treatments.
RESPONSE 10
I would suggest not. The main purpose of intention to treat analysis is to
eliminate any attempts at subversion of the randomisation process. As long
as the test and the randomisation were completely independent, in my
opinion, your approach is fine. If the events were of randomisation and
testing were reversed in time, there would not be an issue.
RESPONSE 11
The approach you are suggesting is one that I have come across in trials
considering the use of antibiotics to treat patients with cholera. The
patients are randomised in to the trial based on a clinical diagnosis of
cholera, and then based on the results of a laboratory analysis patients
remain in the trial and are followed up if the result confirms cholera or
are excluded and follow up stops immediately if the result is negative. I
think whether there is a problem with this approach depends on the question
you are trying to answer.
If clinicians have to make a diagnosis and decision before lab tests then
the results of your trial may become too far removed from clinical practice
to be useful. If you are only interested in lab confirmed and for patients
with a negative result the initial diagnosis and treatment received would
not interfere with the treatment they are switched to then I think it is
probably ok. Could you follow everyone up for the duration of the trial
once randomised, and then you would have more flexibility in the analysis
by presenting results for everyone and subgroup results for those lab
confirmed? Or are the logistics and expense of this too difficult?
RESPONSE 12
We use a similar approach in our analysis of Parkinson's Disease trials
Physicians (even neurologists can misdiagnose PD), and so we end up with
some patients being treated for PD when in fact they don't have it. We get
around this by defining a modified ITT population which is ITT without the
misdiagnosed patients. As far as I understand it the FDA have accepted
this from us so far. Of course we specify the modified ITT population in
the Analysis Plan up front.
RESPONSE 13
You are correct in your approach, don't worry. Altman is talking about
patients who do not get the treatment to which they are allocated. This is
often because of some patient-related cause, e.g. the patient refuses at
the last minute, the patient dies before treatment can begin, the patient
has some clinical event which makes the allocated treatment inappropriate.
However, it is OK to exclude patients who do not have the disease you are
trying to treat, provided this diagnosis does not relate to events after
randomisation. Here the sample is collected before randomisation and the
lab decision will not be influenced by the patient's progress, so it is OK.
RESPONSE 14
I wonder about that 'objective' rule for selection. An old 'war horse'
friend of mine once said, 'there ain't no such thing as objective.'
If you have to define your group in the manner described, I suggest that
the person who selects them not be privy to the test protocol used on each
patient. Perhaps this would qualify as 'triple blind,' but the caution
would remain the same.
RESPONSE 15
When you say it is not possible to delay randomisation until after the test
result, surely this is a case of timing-being the problem. Randomisation
does not need to happen for all patients at the same time. The inclusion
and exclusion criteria determine who can be randomised, and this depends on
the diagnostic test result. It can be that each patient has their own
start time for the drug treatment, provided this is recorded- in case there
is some seasonal difference over time in response. The date the drug was
given can, if necessary be allowed for in the analysis.
Is it ethical to start the treatment - before randomisation?? -if the
patient does not need the drug?
From a statistical point of view you want both groups to be as equal as
possible to truly be able to demonstrate a difference in drug treatment
(i.e equal numbers in both groups). If you randomise before the test
result, you have no control about how many will drop out from one group
because they turned out not to have the condition.
RESPONSE 16
I am sure that my book is just one of numerous publications that discuss
intention to treat analysis. I do not wish the ITT analysis to be labelled
as being due to me. Further, a deeper examination of ITT issues would be
found in one of the many excellent books devoted to randomised trials. That
said I do indeed favour ITT analyses.
In the 10 or more years since I wrote that section I have realised that one
can analyse all randomised patients in a limited number of trials. There
are various constraints, most obviously loss to follow up.
The situation described is not unique. Sometimes the diagnosis is not
available until after the treatment has been given and it may make sense
(in my opinion) to include all patients in the analysis. After all, in
clinical practice one will be faced with the same problem of not being to
wait for a confirmatory diagnosis before treating the patients. Sometimes,
as in the case described, it is possible to stop the treatment as soon as
one discovers that a patient is ineligible (actually that term is not
really correct - they are eligible to be treated).
The action deemed appropriate in a given situation may also be somewhat
affected by the likely proportion of 'ineligible' patients.
In general I think that one should not make avoidable exclusions. However,
I think it is acceptable in some circumstances to make retrospective
exclusions if these are pre-specified in the protocol and if one can be
absolutely sure that no bias is introduced (eg the person(s) deciding would
have no access to any clinical data at or beyond entry to the trial).
Further, in the case described it sounds as if there are clear clinical
reason to take new diagnostic and therapeutic steps for those patients who
are found not to have the target disease. A consequence of this approach is
that the comparison being evaluated relates to a subset of patients who
cannot be identified at the time when the doctor has to initiate treatment.
RESPONSE 17
Clearly the primary endpoint is a comparison within the diagnosis-confirmed
subgroup. Ethics must override the statistical ideal of confirming
diagnosis before randomisation. Presumably it is not appropriate to
continue the randomised treatment for ineligible patients; however
regulatory authorities might be interested in data for these patients if it
can be collected for the study duration . . . Also to demonstrate the
objectivity of eligibility decision rule, it might be better to have
someone who is blind to make the diagnosis eligibility decision.
|
