I would like to thank Stephen Senn and Trevor McMullan for their responses
to my recent query. Their suggestions are given below
>From Stephen Senn:
> Recently a researcher at Salisbury approached me for advice. A paper he
> had submitted for publication in a journal had been rejected on
> statistical grounds. The researcher had performed repeated paired t-tests
> and the referee had suggested that a repeated measures ANOVA would
> be more appropriate. I have little knowledge of mixed models and am not
> sure as to how such an analysis could be applied to his data set. I would
> be grateful for any comments you might have.
Personally I hate repeated measures ANOVA and share the view of two
eminent statisticians: Yates and Finney, that the split plot approach
is not adequate for analysing repeated measures over time. I suspect
that the referee doesn't know what (s)he is talking about.
> The researcher had conducted a retrospective study of stroke and MS
> patients (n=111 and n=21 respectively) with a dropped foot who had
> been treated using the Odstock Drop Foot Stimulator (ODFS). This
> device had been fitted to the patients’ leg with Functional Electrical
> Stimulation (FES) being used to stimulate the muscles as the patient
> walked. The patients were allowed to take the ODFS devices home to
> use as desired. Patients were routinely assessed at their initial visit,
> at 6 weeks, 4˝ months and every 6 months thereafter whilst they
> continued to use the stimulator. At each assessment the patients’ PCI
> (Physiological Cost Index) and walking speed were measured.
> PCI (beats/m) = change in heart rate from rest to walking (beats/min) /
> walking speed (m/min).
> At each assessment patients were asked to walk briskly over a 10 metre
> course six times with approx. 10 seconds rest between walks. During three
> of the six walks the stimulator was activated. The order of the
> (S) / non-stimulated (NS) walks was varied to compensate for fatigue. The
> following order was set: NS S S NS NS S. The mean walking speed and
> PCI for the NS and S runs were calculated. Some patients were not able to
> complete all six runs and their mean readings were therefore based on
> fewer runs. All data were obtained from patient notes. Data for each
> individual run were recorded in the notes, but only the patient means
> have been recorded on the database and there is no way of knowing the
> number of observations on which each mean was based.
I would have preferred a randomised order. This sounds like a
series of repeated n-of-1 trials. As such the data are interesting
and valuable and worth analysing used random effect models. Are you
and this researcher interested in a collaboration?
> For the paper in question only the initial and 4˝ month assessments were
> considered. The ability to walk at least 10 metres was defined as an
> inclusion criterion, as was the availability of data at both time points.
> The researcher had reported the results of repeated paired t-tests on the
> mean values as follows
> NS3 – NS1 (non-stimulated (3rd visit (at 4˝ months)) – non-stimulated (1st
> S3 – S1
> S1 – NS1
> S3 – NS3
> S3 – NS1
> The results were reported separately for stroke and MS patients. No
> formal comparisons were made between the two patient groups.
That seems not unreasonable.
> As you can see it’s a bit of a mess. I have reservations about performing
> an analysis treating mean values as individual observations, especially
> when the means in question are based on varying numbers of individual
> runs. Also, the runs themselves cannot be considered independent. The
> systematic way in which the order of runs was determined also raises
> concerns. Even if it were acceptable to treat the means as individual
> observations, how could a mixed model be applied? Note that S1, S3,
> NS1 and NS3 are all recorded on the same patient.
> All comments gratefully received - even if they’re to say that you don’t
> think that there is any appropriate analysis for this data set without the
> raw values.
> P.S. Other studies have reported that, if the ODFS is used periodically,
> the patients' ability to walk without the ODFS can also improve, hence
> the non-stimulated readings at each follow up assessment.
Professor Stephen Senn
Department of Statistical Science &
Department of Epidemiology and Public Health
University College London
Room 316, 1-19 Torrington Place
LONDON WC1E 6BT
Tel: +44 (0) 171 391 1698
Fax: +44 (0) 171 813 0280
Email: [log in to unmask]
Whilst I am quite definitely no expert it does appear to be a split-plot in
time model that is being suggested. Another technique is Ante
Dependent Analysis. One name you might want to check out is Ron
Kennett, I think of Oxford who is due to have a book published on the
subject. I assume the editor was thinking that you might create a factor
type variable for your index, as ANOVA tends to be a designed
experiment with factors; how you would use a covariate in this context,
I am not sure.
I'd be interested in your replies.
Quintiles Toxicology Pathology Services
Tel: +44 (0)1531 634121
Fax: +44 (0)1531 634753
email: [log in to unmask]