The results of an interim analysis, planned or unplanned, cannot be made
available to anyone who has direct patient contact or to anyone who is
liable to spill the beans (or even a bean) to anyone who has direct patient
contact. Thus programmers are welcome to play with the data; besides, they
shoud be running a qa on an ongoing basis. See, A Manager's Guide to Design
and Conduct of Clinical Trials, Wiley-Liss, 2002.
----- Original Message -----
From: Groeneveld, Jim <[log in to unmask]>
To: <[log in to unmask]>
Sent: Wednesday, February 18, 2004 8:19 AM
Subject: Supposed effects of interim analysis on final analysis
Hi friends,
Currently a hot topic here: what actually is the influence of any unplanned
interim analysis (IA) (in a clinical study) on the later final analysis
(FA)? Just by intuition I do not have so much reservations as apparently
other statisticians may have in doing IA if a final statistical analysis
plan (SAP) already is fixed and FA itself won't be touched in any way, incl.
its results and conclusions, because these would not be different from those
without IA.
However, I can imagine that IA does have concrete influence on the remaining
sample size or the contents of the SAP, especially if the SAP still has to
be made. Consequences from IA may be to continue a study as planned, to
increase the sample size somewhat (just looking at means and sd's), to
(soon) stop a study (due to rather favourite effects), or to cancel a study
completely (because its goals apparently will not be reached by far). It may
also depend on the contents of the IA as opposed to FA. I have the following
questions:
* during SAS program developement (for the FA) the programs usually
are being tested on real, but partly data, with which you can see results
while studying those in order to find possible programming errors and the
like. How does this compare to IA?
* if a SAP already has been written and finalized, would making
(interim) data available for data mining (DM), being combined with other
data, influence the FA in any way? Would it decrease chances of finding
significance with FA if one would not let the FA being influenced at all and
stick to an already fixed SAP?
* what kind of IA would possibly influence FA? Are the contents of IA
identical to those of FA or are they completely different or only viewing
means and sds in order to see whether the study progress (sample size) is on
course? Would emerged and possibly adapted, newer (larger or smaller) sample
sizes touch the SAP and the FA largely? What if the ss would change? Some
influence at least could be observed, but how would one "correct" for that?
Some alpha correction? What weight would it have?
* if IA, then when? At least when a sufficient ss for safety has been
reached? Why that condition? Some reasonable percentage fo the initially
estimated ss?
As can be understood it all is about error to be made during the various
analyses, and if one does multiple dependent tests one should take care for
too much error by adjusting alpha levels in some way. But IMHO IA and FA are
not so much multiple tests, of which the results would have to exist next to
each other. Instead if the FA has been done the IA results are outdated,
overruled, replaced, thrown away and have no value whatsoever and actually
do not exist anymore. Even more, if just doing IA in order to see whether
the assumptions for the initial sample size calculations still are met
(means and sd's) or not (in which latter case ss adaptations could be made)
one does not carry out a significance test (though that could be done).
And if doing IA would require the alpha levels in the FA to be adapted then
I would say they might not even be .05 in the IA as well, because nothing
would be left for the FA. But I would rather regard the alpha levels in the
IA to be exactly or virtually 0, making the IA exploratory only and leaving
the full alpha levels available for the FA.
I have had quite some discussions with other statisticians regarding this
issue. I have not seen any theory regarding this in the past. My question to
you all is, what your opinion is on the consequences of IA for FA and
whether you can point me in the direction of (theoretical) references.
Regards - Jim.
--
. . . . . . . . . . . . . . . .
Jim Groeneveld, MSc.
Biostatistician
Science Team
Vitatron B.V.
Meander 1051
6825 MJ Arnhem
Tel: +31/0 26 376 7365
Fax: +31/0 26 376 7305
[log in to unmask]
www.vitatron.com
[common disclaimer]
|