Dear Linda (and SPMers),
> The significance, referred to as alpha, is the probability of a Type I
> error, i.e., the probability of rejecting H0 when it is true. Let
xalpha
> be the quantity corresponding to the significance level. For example,
if
> a
> test of means is being conducted, xalpha will be the value of the
mean,
> above which the null hypothesis is rejected. xalpha is the value used
to
> make the decision on whether to accept or reject the null hypothesis
for
> the desired significance level. In the distribution of x, there will
be a
> mean value of x, which will hold if the null hypothesis H0 is true.
Call
> this xH0.
>
> Now suppose the null hypothesis is false. Then the distribution of x
has
> a
> mean which is unknown. The experiment provides a value of x which we
will
> call xexper. If xexper is greater than xalpha, the null hypothesis
is
> rejected. The effect size is xexper minus xH0. It is a measure of how
> different the experimental result is from the hypothesis.
Well, let me start explaining from here. When the null hypothesis is
false, then your Xexper follows a distribution known as a non-central
T-distribution. From the name "non-central" T-distribution, people tend
to think that this is basically a T-distribution with its mean shifted.
But this is not the case; it is skewed, and has a slightly different
shape than a regular T-distribution. This deviation from a regular
T-distribution can be summarized by a single parameter known as the
non-centrality parameter. Larger this parameter, more skewed the
non-central T-distribution becomes. The non-centrality parameter depends
on the DF; more subjects you have, larger the non-centrality becomes,
and more power you have in your experiment.
Now, the effect size is a number which summarizes this non-centrality
parameter. It usually describes how large the effect is relative to the
standard deviation (e.g., Cohen's d). This effect size is a convenient
way to describe the effect since it does not depend on DF unlike the
non-centrality parameter.
> xexper is drawn
> from an underlying population of experiments. The distribution, for
an F
> test, is the noncentral F distribution, which requires a knowledge of
the
> noncentrality parameter. Neither Matlab nor SAS supports the
computation
> of the noncentrality parameter. Numerical Recipes states that you
have to
> conduct a Monte Carlo simulation to get the true, underlying
distribution.
Well, the effect size is typically calculated based on the mean and
standard deviation you observe in your pilot data, or from the
literature. You can find the formulae for non-centrality parameters (and
detailed explanation) in the documentation for SAS PROC POWER.
People use Monte-Carlo simulations because this is another way of
calculating power empirically without relying on formulae.
> Is this different for a two-sample t-test?
Well, you can calculate the effect size for a two-sample t-test based on
mean and standard deviation.
Good luck!
-Satoru
|