Hi friends,
Currently a hot topic here: what actually is the influence of any unplanned interim analysis (IA) (in a clinical study) on the later final analysis (FA)? Just by intuition I do not have so much reservations as apparently other statisticians may have in doing IA if a final statistical analysis plan (SAP) already is fixed and FA itself won't be touched in any way, incl. its results and conclusions, because these would not be different from those without IA.
However, I can imagine that IA does have concrete influence on the remaining sample size or the contents of the SAP, especially if the SAP still has to be made. Consequences from IA may be to continue a study as planned, to increase the sample size somewhat (just looking at means and sd's), to (soon) stop a study (due to rather favourite effects), or to cancel a study completely (because its goals apparently will not be reached by far). It may also depend on the contents of the IA as opposed to FA. I have the following questions:
* during SAS program developement (for the FA) the programs usually are being tested on real, but partly data, with which you can see results while studying those in order to find possible programming errors and the like. How does this compare to IA?
* if a SAP already has been written and finalized, would making (interim) data available for data mining (DM), being combined with other data, influence the FA in any way? Would it decrease chances of finding significance with FA if one would not let the FA being influenced at all and stick to an already fixed SAP?
* what kind of IA would possibly influence FA? Are the contents of IA identical to those of FA or are they completely different or only viewing means and sds in order to see whether the study progress (sample size) is on course? Would emerged and possibly adapted, newer (larger or smaller) sample sizes touch the SAP and the FA largely? What if the ss would change? Some influence at least could be observed, but how would one "correct" for that? Some alpha correction? What weight would it have?
* if IA, then when? At least when a sufficient ss for safety has been reached? Why that condition? Some reasonable percentage fo the initially estimated ss?
As can be understood it all is about error to be made during the various analyses, and if one does multiple dependent tests one should take care for too much error by adjusting alpha levels in some way. But IMHO IA and FA are not so much multiple tests, of which the results would have to exist next to each other. Instead if the FA has been done the IA results are outdated, overruled, replaced, thrown away and have no value whatsoever and actually do not exist anymore. Even more, if just doing IA in order to see whether the assumptions for the initial sample size calculations still are met (means and sd's) or not (in which latter case ss adaptations could be made) one does not carry out a significance test (though that could be done).
And if doing IA would require the alpha levels in the FA to be adapted then I would say they might not even be .05 in the IA as well, because nothing would be left for the FA. But I would rather regard the alpha levels in the IA to be exactly or virtually 0, making the IA exploratory only and leaving the full alpha levels available for the FA.
I have had quite some discussions with other statisticians regarding this issue. I have not seen any theory regarding this in the past. My question to you all is, what your opinion is on the consequences of IA for FA and whether you can point me in the direction of (theoretical) references.
Regards - Jim.
--
. . . . . . . . . . . . . . . .
Jim Groeneveld, MSc.
Biostatistician
Science Team
Vitatron B.V.
Meander 1051
6825 MJ Arnhem
Tel: +31/0 26 376 7365
Fax: +31/0 26 376 7305
[log in to unmask]
www.vitatron.com
[common disclaimer]
|