I have fallen into a conversation about Sampling, Confidence Intervals
(CIs), Point Estimates (PEs) and sample sizes (N).
The main question is are probability samples always better than
non-probability samples. This is an issue because many sociologists use
non-probability samples for qualitative research. Some are wondering
whether this is okay, or so flawed as to be denounced. So, that's the
set-up.
One position in this discussion is that regardless of population size,
probability sampling is needed. With probability sampling your point
estimates are expected to, on average over repeated samples, center on the
true population parameter value. This position also says you can
calculate confidence intervals as long as you can calculate the standard
error. So, for example, even if sample size=5, you should be able to
calculate the standard error (in the silly case of estimating a simple
regression on 5 cases, say).
Another position accepts that point estimates are good, but maintains that
statistical theory cannot justify the construction of CIs if the sample
size is small. This position maintains that you need a minimum sample
size of 30 for the CIs to be defensible (even for unadjusted means (e.g.,
X-bar)).
A third and fourth position contendss that probability samples are
completely unnecessary. Position three says that if one wants to study a
case just pick the case. (Positions 1 and 2 agree, but assert that one
can make NO generalizations on the basis of that case). Position three is
that one can generalize "theoretically". What this means, and how it
differs from generalizing to population parameters, remains unarticulated
at this time.
The fourth view is that one should use one's information to draw a
non-probability sample. This, the fourth position maintains, is the
Bayesian approach--pick the sample based on your knowledge.
So, I would welcome any of the following:
1)Comments on the accuracy of the above claims and positions.
2)Articulation of new points in that discussion, as well as new positions.
3)Suggestions for literature on some of the key points (most notably, what
is the minimum sample size to justify calculation of CIs).
4)Anything else of interest.
Thanks a bunch.
Sam
|