 |
 |
Re: Web site Hello, This message was originally put to the Podiatry mail base. The author and I would appreciate any advice that is forth coming from the members concerned with Evidence based health. Thank you in anticipation, Susan Stacpoole-Shea Ballarat, Victoria, Australia Dear all, I'd like to pose a thorny question for the researchers on the mailbase. This question was initially bought to my attention by Lloyd Reed from QUT a couple of years ago and has stimulated much debate among my colleagues at UWS. My apologies for the length of this posting (the text is taken from a paper I've started writing). In many fields of biomedical research, information is collected on multiple joints or organs from the same subject. For example, many ophthalmology studies record data from both eyes, and in the case of foot and ankle research, data is often collected from both feet. This raises a significant, yet largely overlooked problem when it comes to statistical analysis. One of the fundamental requirements of statistics is that each data point must represent an independent observation to justify being considered a "unit". In most cases, the unit of measurement is the subject, so if, for example, 50 subjects are enrolled in the study, each observation recorded from each subject counts as a single unit, ie: n=50. However, if data is recorded from both feet, a major problem arises. What is the unit of measurement – a subject, or a foot? Do we have a sample of n=50 people, or a sample of n=100 feet? A cursory examination of the foot and ankle literature reveals dozens of examples of statements like "We recruited thirty subjects (sixty feet)". From a conceptual viewpoint, it does seem a little odd to conduct research into individual feet rather than people, as clearly the way an individual foot functions is dependent on the person attached to it. For example, the healing rate of a surgical wound is strongly dependent on its blood supply, the pressure distribution under a foot is strongly dependent on the gait pattern of the individual, and the pain experienced following local anaesthetic injection strongly dependent on the individual’s pain threshold. In each of these examples, it is likely that the degree of association between right and left feet in the same subject would be far greater than the association between different subjects. Therefore, if both right and left feet were counted as single independent observations, the researcher is essentially "double-dipping" their data, ie: counting each subject twice. Doing so will increase sample size and decrease variability in the data, thereby increasing the power of the study and increasing the likelihood of detecting statistical differences. But are these "significant" differences real? In order to demonstrate how the decision to pool or not pool right and left foot data can influence results, I have developed a dataset of "dummy" data for 30 subjects (see below). For the purpose of discussion, the data can be considered to represent rearfoot motion values (in degrees) for 30 subjects with and without foot orthoses for both right and left feet. Paired t-tests were then used to compare the "without orthosis" and "with orthosis" conditions for the right foot only, the left foot only, the average of the right and left feet, and with right and left foot data combined. Key to table: WOOR - without orthosis right foot, WOOL - without orthosis left foot, WOR - with orthosis right foot, WOL - without orthosis left foot WOOR WOOL WOR WOL 1 2 1 3 2 2 4 4 2 2 3 6 6 4 3 4 8 7 2 2 5 4 4 1 1 6 5 5 4 3 7 6 6 5 4 8 3 3 7 7 9 2 1 5 5 10 4 3 3 3 11 5 5 2 2 12 7 7 4 2 13 4 3 3 3 14 2 1 1 1 15 2 2 1 1 16 2 1 3 2 17 4 4 6 2 18 6 6 4 3 19 8 7 2 2 20 4 4 6 5 21 5 5 4 3 22 6 6 5 4 23 3 3 7 7 24 2 1 5 5 25 4 3 3 3 26 5 5 2 2 27 7 7 4 2 28 4 3 3 3 29 2 1 1 5 30 2 2 1 1 For right foot data, there was no difference in rearfoot motion with or without foot orthoses (t29=1.83, p=0.077). Similarly, for left foot data, there was no difference in rearfoot motion with or without foot orthoses (t29=1.68, p=0.104). For the averaged data, there was no difference in rearfoot motion with or without foot orthoses (t29=1.82, p=0.079). For pooled right and left foot data (thereby increasing the sample size from 30 to 60), the paired t-test revealed a significant reduction in rearfoot motion when wearing foot orthoses compared to the without orthosis condition (t59=2.44, p=0.018). The results of this simple example clearly highlight the problems inherent in analysing pooled data from paired limbs. In the example provided, foot orthoses had no effect on rearfoot motion on the right foot when analysed in isolation, no effect on the left foot when analysed in isolation, and no effect when the two feet were averaged. However, when the right and left data was pooled, a significant reduction in rearfoot motion was apparent in the orthosis condition. Although the difference was small in absolute terms, there is little doubt that such a difference would be reported as a "significant" finding. Thus, depending on whether data is pooled or not, it could be concluded that foot orthoses either do influence rearfoot motion when walking or they do not. So my questions are as follows: What is the best approach for the statistical analysis of paired data? If we decide to analyse one foot only, which foot do we pick (and why)? Are there any situations in which analysing paired data is justifiable? Kind regards, Hylton Hylton B. Menz