I have quoted for some years now what I believed to be a teaching of
Deming, that in some circumstances non-random sampling was preferable.
In particular, when a small "representative" sample from a
non-homogeneous population is needed, a judgment sample gives more usable
results. The "cost" is that you cannot make probability estimates, but
with small samples these are likely to be of little practical use.
I have quoted this as based on "Out of the crisis" 1987, but on looking
again for the source found only the contrary: p353, "The only safe plan is
use of random numbers for selection of items from a lot." (Attributed to
Mr Dave West, 1982)
What I then re-found was Deming's 1960 book "Sample Design in Business
Research", p31, "it is well to mention the existence and occasional
utility of judgment samples, deliberately biased samples, and chunks.
... For an evaluation of the reliability of such a survey, we must rely
on the expert's judgment: we can not use the theory of probability.
... There is no way to compare the cost of a probability sample with the
cost of a judgment sample, because ... [c]ost has no meaning without a
measure of quality."
One question is therefore whether any Demingists in the audience know
whether he changed his attitude over the years and became more puritan
with respect to statistical methods, or whether he continued to talk
about non-probability sampling but omitted it from "Out of the crisis" as
not relevant to quality-control sampling.
The second point is to ask if other list members teach this heterodox
view. I like to pose to students the question: if you are studying
education in, say, Hull and Hong Kong, and can study ONE school in each
city, how would you choose the schools? Simple random sampling from a
frame made up of all the school names is clearly ineffective - if you
solemnly then compared the small Moslem school in Hull with the largest
comprehensive in the New Territories, who would read the report, except
for the laughs?
R. Allan Reese Email: [log in to unmask]
Associate Manager Direct voice: +44 1482 466845
Graduate Research Institute Voice messages: +44 1482 466844
Hull University, Hull HU6 7RX, UK. Fax: +44 1482 466846
If Stephenson, Marconi and Edison had lived in the opposite order, so
the mobile phone got invented before the railway, would people still
find the need to travel, and tell people where they are?