Greetings.
I read with interest the summary that Dietrich Alte
posted on Wed, 2 Jan 2002 of the responses to his
query of Fri, 14 Dec 2001.
However, not discussed were aspects of this which
have concerned me for some time. These turn on the
following considerations, which should be read in
the context of the "presecriptive" algorithms, of
the kind which recommend P-values of the order of
0.1, 0.2, 0.25 for inclusion/retention of covariates
in the model, etc.
1. Although these procedures are "standard practice"
for data where "many" covariates have been recorded,
and one is a bit short of ideas as to what depends on
what and how, there is a definite question of *how many*
covariates there should be before such methods become
appropriate, and/or whether these threshold P-values
should depend on the number of covariates.
To see that this is a definite question, consider
data where there is a response variable, and a
single covariate X only. The idea that a threshold
P-value of the order of 0.1-0.2 should determine
whether X gets included, is obvious nonsense
relative to the way this would normally be done.
Or, you could say: Yes, we use the same method,
but the P-value is now, say, 0.05 (or 0.01, or
whatever you want to use in a test for "no effect").
[i.e. we include the variable anyway; then we
test for its effect, i.e. for whether or not it
survives.]
If, therefore, such a method is appropiate for
larger numbers of covariates, then there must be
a line somewhere: fewer covariates than this,
and you wouldn't do it that way; more, and you may.
Or, you do it that way all the time, but the
P-values depend on the number of covariates.
QUESTION: why don't we see this discussed?
2. (Related, but introducing a new dimension)
If any of the covariates has the slightest
effect, then with enough data this effect will
be detected (so long as it is not aliased with
some other). The "detection method" might, for
instance, be a confidence interval obtained
after fitting all the covariates.
On the other hand, in real life we are usually
stuck with the data we have. Specifically, we
have a multivariate list of covariate values
(along with the associated values of the response).
For a particular covariate X, in the context of
particular values for the coefficients of the
other covariates, there is a trade-off (also
known as the Power Function) between the P-value
at which an effect due to X can be detected, and
the size of the effect (i.e. the value of the
coefficient of X).
Using a prescriptive P-vaue (e.g. 0.2) as a
criterion for inclusion of X in the model
therefore evokes an implicit size of effect.
The question behind the scenes here is: Given
the list of covariate values, how big does the
effect of X have to be, before this method is
likely to lead to its inclusion?
It might be very small, (i.e. the covariate
list implies high power for X), or very large
(so very low power for X). If the power is high,
then you might as well include X anyway since,
if it does have an effect then you are almost
certain to detect it. On the other hand, if
the power is very low then why bother? Your data
are probably inadequate to ascertain the effect of X.
And what do "small" and "large" mean? Well,
that of course depends on NOT being "short
of ideas as to what depends on what, and how".
It seems to me that considerations of Power
are essential in considering what variables
might be included in a model.
AND YET: In the dicussions (I include the
literature) I see little consideration of this.
(Note: for a single covariate X the Power
can of course be calculated from the values
of the covariates alone, ignoring the observed
responses. For X in company with other covariates,
it may be necessary to use observed response values
to get reasonable values of the other coefficients
to use as context for calculating the Power for X.)
==============================
Comments? (I'll happily summarise if sufficiently
interesting).
Best wishes to all,
Ted.
--------------------------------------------------------------------
E-Mail: (Ted Harding) <[log in to unmask]>
Fax-to-email: +44 (0)870 167 1972
Date: 31-Jan-02 Time: 18:12:40
------------------------------ XFMail ------------------------------
|