Here are some responses to my question about Randomised trial? My
original message comes last. Excuse for the long posting.
Weaknesses in the manuscript, the vage description of the randomisation method
and the similar distribution between the treatment arms, gave me gutfeelings
that something was wrong. I assumed that the randomness in a small sample would
result in larger differences than what was seen.
Some responses agreed to my impression, others did not.
Victor Montori gave one explanation on what method that may have been used for
randomisation:
>Sounds like they used a random number generator and matched it with the last
>two clinic numbers - for instance - the random number generator gives 21 54
>23 65 72 65 / 86 87 23 35 65 23 and you say that if the last two digits of
>the record number are in group on the left then they go to intervention, if
>on the right they go on placebo.
Ian Reeves provided a reference to the litterature:
>The best work i have found on this is
>Altman (1985) The Statistician 34, pp 125-136
>Comparability of randomised groups.
Joël Mossing shared my concern that the distributions looked suspiciously
similar for randomness.
He gave a reference too Fisher that showed that Mendel's data was doctored
because it fitted the hypothesis too well. He proposed that the issue could be
solved by simulation techniques.
>I find your question very interesting. It reminds me somewhat of Fisher who
>tried to show that Mendel's data was doctored because it fitted the
>hypothesis too well. Have a look at
>http://www.nyu.edu/classes/murfin/scihistoryboard/messages/286.html
>for a discussion. I also find that the 2 the populations are a bit too
>similar for randomness. But how you could check it formally is another
>thing. I suppose one way is to do a kind of Monte Carlo Simulation, by
>trying to generate thousands of binomial (or multinomial) tables for each
>factor and then see what proportion of those generated tables is more
>extreme than the table you were given (assuming that all factors are
>independent ...).
Matthew Zacks proposed an elegant method to test if the distributions were
random:
>For the six variables whose counts by treatment you have
>provided, the Fisher-Irwin 2-tailed exact test significance probabilities
>are the following: 1.000, 1.000, 1.000, 0.104, 0.402, 0.691.
>If the distributions of these variables were too evenly distributed, then
>one would expect the distributions of these significance probabilities to be
>clustered near 1.000. The reference distribution would be the uniform
distribution.
>One method to compare the observed values to that expected from the
>reference distribution is to use the Smirnov exact test or the
>Kolmogorov-Smirnov one sample test.
>This comparison shows no statistically significant differences between the
>observed distribution and the uniform distribution:
> 0.104 0.402 0.691 1.000 1.000 1.000 Observed distribution
> 0.167 0.333 0.500 0.667 0.833 1.000 Reference distribution
>Therefore, I don't think you have enough evidence to say that the
>distributions of the six variables were too even.
My conclusion is that doubts may be justified when distributions are too similar
but in this case there is not enough evidence to prove that randomisation was
faked.
I have accepted the work and do not intend to write an invited comment.
Original message:
> I have been invited to comment on a manuscript reporting the result from a
> randomised trial of a treatment of sepsis after GI perforation. I have doubts
> about the randomisation. The authors describe that randomisation was made
"using
> the last two digits of the patients inpatient number and random number table".
>
> In my opinion the randomisation resulted in an unlikely even distribution over
a
> number of variables, see table. I have therefore expressed doubts about the
> study being truly randomised.
>
> I would like to receive comments from others. How likely is it that a
> randomisation of patients would result in such an even distribution over
> factors?
>
> Factor Treatment 1 Treatment 2
>
> Male 17 16
> Female 1 2
>
> Site of perforation
> Gastric 1 2
> Duodenal 6 5
> Enteric 11 11
>
> Duration of symptoms
> <1 day 15 14
> >= 1 day 3 4
>
> Hematocrit
> Normal 18 14
> Deranged 0 4
>
> Serum Potassium
> Normal 13 16
> Decreased 5 2
>
> Serum protein
> Normal 15 13
> Low 3 5
>
> Roland Andersson
>
> //\\//\\//\\//\\//\\//\\//\\//\\//\\//\\//\\
>
> Roland Andersson, MD PhD
> Department of Surgery
> County Hospital Ryhov
> S-551 85 Jönköping
> SWEDEN
>
> e-mail: [log in to unmask]
> phone: +46-36-321344
> fax: +46-36-321321
> ____________________________________________
> //\\//\\//\\//\\//\\//\\//\\//\\//\\//\\//\\
|