On 21/08/2011 12.37, Stephen Senn wrote:
> Multiplicity is tricky issue. I too do not believe in the mystic value of pre-specification. Nevertheless evidentially there are some different scenarios one can imagine.
> 1. Several outcomes were measured all were analysed and presented.
> 2. Several outcomes were measured all were analysed and presented and one was prespecified.
> 3. Several outcomes were measured but only the pres-specified one was presented and it was always known that this would be the case.
> 4. Several outcomes were measured and analysed but one that was not pre-specified was presented.
It seems to be case 2.
The report on the NEJM reads: "The predefined primary outcome was the
change from baseline in ultrasonographic measurement of the mean
carotid-artery intima–media thickness, which was defined as the average
of the means of the far-wall intima–media thickness of the right and
left common carotid arteries, carotid bulbs, and internal carotid
arteries in the two study groups.". What seems to have been the previous
primary outcome (Change in ultrasound-determined average carotid artery
plus average common femoral artery intima-media thickness) is then
presented as a secondary outcome.
This is the same information present on clinicaltrials.gov: however the
trial was registered on clinicaltrials.gov only in october 2007, while
it had been carried out between June 2002 and April 2006. Furthermore,
the panel that allegendly suggested to change the study aim was held one
month later than that, in November 2007. The data were last changed on
June 19, 2008 (two months after the publication of the results.
Surprisingly, clinicaltrials.gov doesn't keep trace of changes, and
unfortunately the wayback machine on web.archive.org has no captures of
that page before January 2008.
The problem, though, is that no account is given of the fact that what
constitute the "predefined primary outcome" was actually changed before
registering the trial. Thus, the sentence published on the registry is
ambiguous (it does not say whether it was predefined) and that published
on the NEJM is false, as it reports it as "predefined" without
mentioning the change. It is like having a history book saying that
Colombo left Palos to find America.
It seems that all this has little effect on the interpretation of the
results (negative anyway), still it denotes a disregard for giving a
fair account of what happened which mines the thrust also on other
aspects of the story, including the reasons for delaying its publication.
For example, to come to your question on whether the data were still
blinded, it seems unlikely that in November 2007 there were still 40,000
images to be manually processed, as reported to support the view that
they were, when the paper (online 30 March 2008, in print 4 April, no
information on when it was submitted or accepted) could not have been
reasonably submitted later than January 2008.
regards,
Piersante Sestini
|