Dear list members,
I contacted GenStat tech support regarding my problem of calculating a
chi-squared test statistic for the variance component of the meta-analysis
model that I've been trying to set up (a Z-statistic using the variance
component and SE is not the best because variances do not follow a normal
distribution). I received the following reply.
*************
Intead of the chi-squared test, a likelihood ratio test is usually
recommended, ie. comparing the deviance (-2*logRL) of the model
containing the variance component to one omitting the component.
Dropping one component gives a test based on 1df (chi-squared
distribution) if it is unconstrained. If it is constrained positive,
then the test statistic is based on a 50:50 mixture of chi-squared
distributions on 0 and 1 df respectively. This gives a test statistic
of approx 2.7 for p=0.05.
****************
So, I went ahead and tried this out, but I want to check with the list
members to see if I am doing this correctly.
The original intercept-only model, with the variance component included, is:
vcomp [cadjust=none] random=UNITS+STUDY; init=1,1; constr=fix,pos
reml [print=model,components,effects,means,vcovariance,deviance;
weights=INVVAR] D;
This gives a deviance of -4.48 with 18 df.
Now, I drop the variance component:
vcomp [cadjust=none] random=UNITS; init=1; constr=fix
reml [print=model,components,effects,means,vcovariance,deviance;
weights=INVVAR] D;
This gives a deviance of 6.41 with 19 df.
So, I subtract the deviances 6.41 - (-4.48), which gives a value of 10.89,
and I have 19-18 = 1 df.
A chi-squared stat of 10.89 with 1 df gives a P-value of .00097. However,
because the dropped variance was constrained positive, I'm dealing with a
50:50 mixture of chi-squared distributions on 0 and 1 df. So, from my
understanding, that would mean I would need to cut the P value in half.
So, the p-value would be .00049. Thus, the variance component can be
considered significant.
Am I doing this correctly? I'm assuming I need to halve the P-value for the
50:50 mixture, because a normal chi-squared test stat of 2.7 with 1 df has a
P-value of 0.10, not 0.05.
Thank you for your time,
James
James Krieger, M.S.
Science Editor, Pure Power Magazine
http://www.purepowermag.com
Webmaster, WSU Strength and Conditioning
http://www.wsu.edu/~strength
|