Hello,
This message was originally put to the Podiatry mail base. The author and
I would appreciate any advice that is forth coming from the members
concerned with Evidence based health.
Thank you in anticipation,
Susan Stacpoole-Shea
Ballarat, Victoria, Australia
Dear all,
I'd like to pose a thorny
question for the researchers on the mailbase. This
question was initially
bought to my attention by Lloyd Reed from QUT a
couple of years ago and
has stimulated much debate among my colleagues at
UWS. My apologies for the
length of this posting (the text is taken from a
paper I've started
writing).
In many fields of
biomedical research, information is collected on multiple
joints or organs from the
same subject. For example, many ophthalmology
studies record data from
both eyes, and in the case of foot and ankle
research, data is often
collected from both feet. This raises a significant,
yet largely overlooked
problem when it comes to statistical analysis. One of
the fundamental
requirements of statistics is that each data point must
represent an independent
observation to justify being considered a "unit".
In most cases, the unit of
measurement is the subject, so if, for example,
50 subjects are enrolled
in the study, each observation recorded from each
subject counts as a single
unit, ie: n=50. However, if data is recorded from
both feet, a major problem
arises. What is the unit of measurement – a
subject, or a foot? Do we
have a sample of n=50 people, or a sample of n=100
feet? A cursory
examination of the foot and ankle literature reveals dozens
of examples of statements
like "We recruited thirty subjects (sixty feet)".
From a conceptual
viewpoint, it does seem a little odd to conduct research
into individual feet
rather than people, as clearly the way an individual
foot functions is
dependent on the person attached to it. For example, the
healing rate of a surgical
wound is strongly dependent on its blood supply,
the pressure distribution
under a foot is strongly dependent on the gait
pattern of the individual,
and the pain experienced following local
anaesthetic injection
strongly dependent on the individual’s pain threshold.
In each of these examples,
it is likely that the degree of association
between right and left
feet in the same subject would be far greater than
the association between
different subjects. Therefore, if both right and
left feet were counted as
single independent observations, the researcher is
essentially
"double-dipping" their data, ie: counting each subject twice.
Doing so will increase
sample size and decrease variability in the data,
thereby increasing the
power of the study and increasing the likelihood of
detecting statistical
differences. But are these "significant" differences
real?
In order to demonstrate
how the decision to pool or not pool right and left
foot data can influence
results, I have developed a dataset of "dummy" data
for 30 subjects (see
below). For the purpose of discussion, the data can be
considered to represent
rearfoot motion values (in degrees) for 30 subjects
with and without foot
orthoses for both right and left feet. Paired t-tests
were then used to compare
the "without orthosis" and "with orthosis"
conditions for the right
foot only, the left foot only, the average of the
right and left feet, and
with right and left foot data combined.
Key to table:
WOOR - without orthosis
right foot, WOOL - without orthosis left foot, WOR -
with orthosis right foot,
WOL - without orthosis left foot
WOOR WOOL WOR
WOL
1 2 1 3 2
2 4 4 2 2
3 6 6 4 3
4 8 7 2 2
5 4 4 1 1
6 5 5 4 3
7 6 6 5 4
8 3 3 7 7
9 2 1 5 5
10 4 3 3 3
11 5 5 2 2
12 7 7 4 2
13 4 3 3 3
14 2 1 1 1
15 2 2 1 1
16 2 1 3 2
17 4 4 6 2
18 6 6 4 3
19 8 7 2 2
20 4 4 6 5
21 5 5 4 3
22 6 6 5 4
23 3 3 7 7
24 2 1 5 5
25 4 3 3 3
26 5 5 2 2
27 7 7 4 2
28 4 3 3 3
29 2 1 1 5
30 2 2 1 1
For right foot data, there
was no difference in rearfoot motion with or
without foot orthoses
(t29=1.83, p=0.077). Similarly, for left foot data,
there was no difference in
rearfoot motion with or without foot orthoses
(t29=1.68, p=0.104). For
the averaged data, there was no difference in
rearfoot motion with or
without foot orthoses (t29=1.82, p=0.079). For
pooled right and left foot
data (thereby increasing the sample size from 30
to 60), the paired t-test
revealed a significant reduction in rearfoot
motion when wearing foot
orthoses compared to the without orthosis condition
(t59=2.44,
p=0.018).
The results of this simple
example clearly highlight the problems inherent
in analysing pooled data
from paired limbs. In the example provided, foot
orthoses had no effect on
rearfoot motion on the right foot when analysed in
isolation, no effect on
the left foot when analysed in isolation, and no
effect when the two feet
were averaged. However, when the right and left
data was pooled, a
significant reduction in rearfoot motion was apparent in
the orthosis condition.
Although the difference was small in absolute terms,
there is little doubt that
such a difference would be reported as a
"significant" finding.
Thus, depending on whether data is pooled or not, it
could be concluded that
foot orthoses either do influence rearfoot motion
when walking or they do
not.
So my questions are as
follows:
What is the best approach
for the statistical analysis of paired data?
If we decide to analyse
one foot only, which foot do we pick (and why)?
Are there any situations
in which analysing paired data is justifiable?
Kind regards,
Hylton
Hylton B.
Menz