 |
 |
A small sample size is of course a problem, but there are many many scientific studies in which small samples provide the only current option. 1. With a small sample, it is naturally more difficult to attain statistical significance, A larger effect size is needed. But it is still possible to reach "significance." If one is in the (admittedly unusual) circumstance of knowing prior distribution shape on the main outcome measure (e.g., by examining a large database with the group in question), sample size requirements are diminished, as one can fit a more exact statistical model. If pre-treatment stability can be established (e.g. by repeated measures), needed sample size can be driven down greatly. 2. Generalization to the population of interest is also an issue. Random sampling from the population helps a great deal, and in theory can firmly establish generalization. But usually one cannot prove that one's sample is really random, as one does not have a list of everyone in the total population It is easier for a small sample to be non-representative of the population which one would like to generalize. Repeating the sample in important variants of the target population can to a long way to address generalization concerns. Cheers. -- MVJ On 7/30/2011 8:15 AM, Ted Harding wrote: > Possible answers to this question are very contingent on the > particularities of the object of study. > See comments in-line below. > > On 30-Jul-11 12:35:45, Anoop Balachandran wrote: >> Hi everyone, >> >> How valid is a study which uses a sample size of 12 with a >> within-subject design? Or how well can we extrapolate the >> conclusions to the population studied. > Validity: Regardless of sample size, a properly executed study > is always valid -- as far as it goes. How far it goes depends, > again, on the particularities. Even then, whether it is *adequate* > for your purposes depends on how far you need it to go. It might > be that your needs are so broadly defined that a very imprecise > answer will be adequate, and a small sample may be adequate. > If your needs for precision are stringently set, and if the > variation in the population is large, then a small sample will > not be adequate, and you will need to calculate what sample > size will be adequiate. However, for that you need an estimate > of the variability in the population. > > Extrapolation to population: *Very* contingent on how likely it > is that a sample of given size will capture, "representatively", > the essential features of the variation in the population. > In particular, if the population is "non-homogenous" (i.e. > consists of several groups each with distinct characteristics) > then your sample will need to be large enough to have a good > propbability of including an adequate number from each group. > Or, if you know of markers which can identify the distinct > groups, then you might consider adopting a stratified sampling > approach. > >> I have heard that you need at least 30 per group to get a normal >> distribution. That been said, I have read that even numbers >> approaching 10 can get you close to a normal distribution. > If the distribution of values in the population being sampled > from is close to Normal, then you can validly treat your sample > as a sample from a Normal distribution, regardless of sample size. > However, varifying this condition is another question. People > often carry out some test (e.g. Kolmogorov-Smirnov) for Normality. > It will be important to choose the sample size large enough that > a deperture from Normality in the population which is large enough > to invalidate your results has high probability of being detected > (power of test). > > If you have a sample from a non-Normal population, then in general > a large enough sample size means that statistics calculated in the > analysis (e.g. sample means) will have distributions which are > close to Normal. However, the sample size you would need will depend > on the degree and kind of the departure from Normality. In particular, > if the variable has (say) a positively skew distribution in the > population, then Normality for the (say) sample mean will only > slowly be approached (skewness is particularly toxic to the Normality > of sample means from large samples). > > In certain (admittedly extreme) cases, no sample size however > large can give you any improvement with respect to Normality > of (say) sample means. > > For instance, imagine a machine-gun mounted on a pivot and > able to rotate in a horizontal plane. At some distance to > one side is a plaster wall. The gun is rotated rapidly about > its pivot, and continually fires a very large number of rounds. > The positions of the resulting holes in the wall are measured, > and used as the data. > > Then the distribution of the positions is not Normal, and > no matter how large a sample you take, the distribution > of the sample mean will be exactly the same as that for > the original holes, so will be just as non-Normal. > > For the technically minded: The circumstances of the experiment > are such that the direction in which the machine-gun is pointing > when a bullet is fired is uniformly distributed round a circle. > The distance of the point of impact from the point on the wall > nearest to the gun is the tangent of this angle (multiplied > by the distance from gun to wall). It then folllows that the > points of impact have a Cauchy distribution, which is totally > immune to the Normalising effects of large sample means! > >> The study I am talking about measures muscle fiber size using >> a biopsy and also assess strength measurements. > You need to do a preliminary study in order to ascertain how > these variables are typically distributed. Then you will be > in a position to tackle the other issues discussed above. > With luck, you may find such data in the literature ... > >> In the exercise field, we sill use P-values so they haven't >> reported the confidence intervals. > ... or quite possibly not, if the workers in this field are > so indifferent as to the nature of their primary data. > > However, there may be useful information in physiological > literature of a more serious kind! > >> Thanks! > Hoping it helps! > Ted. > > -------------------------------------------------------------------- > E-Mail: (Ted Harding)<[log in to unmask]> > Fax-to-email: +44 (0)870 094 0861 > Date: 30-Jul-11 Time: 14:15:02 > ------------------------------ XFMail ------------------------------ -- */Mark V. Johnston,/**//**/Ph.D./* Professor, Occupational Science and Technology, College of Health Sciences University of Wisconsin -- Milwaukee (414) 229-3616