...
> I think Alexa was correct: I don't think its matter of df's per se,
> but whether the underlying linear model is correct (and which terms
> in that model are used to construct the F-contrast). If in a
> multifactor design, each of the effect-by-subject interaction terms
> is drawn from the same, single IID error distribution (the "pooling"
> assumption), such that their SSs are proportional to their df's,
> then pooling provides a statistically more efficient estimate of
> that error distribution, so would be preferred.
You are assuming these variance sources to be a homogeneous hamburger,
which can be chopped up in different ways without changing its
substance. This is fine in the randomization logic, and it is correct
within that logic.
But if you consider experiments on subjects as samples from a
population of experiments on subjects, and interpret p values and
inference as generalizations on the population of subjects, this
reasoning no longer applies; you can't infer anything about the
population from scanning three subjects, unless you assume that
subjects don't differ much, and you use session variance as the
relevant model of randomness.
...
> powerful. Fortunately, in my experience, the results of the two
> approaches are usually similar, but pooling also offers an
> additional (orthogonal) advantage of a greater number of df's with
> which to estimate the spatial smoothness of the residuals, which
> reduces the conservativeness of RFT (for low numbers of subjects).
This is where the issue is specifically one of RFT theory. I do not
question your experience, but where was it shown that the smoothness
of F tests obtained with different numerator dfs is the same, given
that they have the same residuals in common? After all, the Fs reflect
the best linear combination of contrasts, and this combination varies
from voxel to voxel. How can their smoothness be the same, if the
dimensionality of the linear combination changes? That means different
chances to change from voxel to voxel. (This is a real question, I do
not know the answer). A permutation test would take care of this
automatically.
> But I would be interested to hear the opinions of a proper statistician!
I have already mentioned my thoughts on this in reply to Alexa...
Very best wishes,
Roberto
> From: SPM (Statistical Parametric Mapping) [[log in to unmask]] on
> behalf of Stephen J. Fromm [[log in to unmask]]
> Sent: 06 January 2011 15:25
> To: [log in to unmask]
> Subject: Re: [SPM] Design matrix for each or all subject(s)?
>
> "In probably rather simple terms - in a pooled error test, if done
> correctly, the df are greater but are valid. They are not
> 'inflated'. The error is also greater - being pooled from what would
> be different partitioned error cells."
>
> The df are not inflated, but only in the sense that they're
> appropriate for that F-test. The crux here is that the F-test is
> the wrong one: that is, the F-test is inappropriate for the
> inference being undertaken. The correct F-ratio has both a smaller
> error term and fewer dof. In that sense, the dof of the "pooled"
> test is inflated.
>
> "If sphericity holds (or is corrected for), this should be just as
> valid as a partitioned error test. I understand it to involve a
> pooled *estimate* of an error variance that may (or may not) be the
> same for all cells - and this estimate is more powerful, with more
> df, if assumptions hold."
>
> I disagree. The classical texts state that the assumption that has
> to hold is that certain interaction terms vanish (in the sense of
> being statistically negligible), which is not true in general. I'm
> not an expert and haven't had time to fully explore the issue, but I
> think sphericity is a different issue.
>
> "If many contrasts per subject are taken to the 2nd level as in the
> 3 subjects/ 12 levels example, and subject effects are not modelled
> (discounted from the error term) as the Flexible Factorial allows..."
>
> It's more complicated than modeling pure subject effects. Rather,
> the variance components at issue are the _interaction_ with the
> effect of subject. This cannot be done correctly using the "pooled"
> method.
>
> -----------
>
> One citation: David C. Howell, _Statistical Methods for
> Psychology_. It's one of the references cited by Henson and Penny
> ("ANOVAs and SPM"), at
> http://www.fil.ion.ucl.ac.uk/~wpenny/publications/rik_anova.pdf
> (Unfortunately, the bibliography/endnote list was cut out
> accidentally in the revised version.) I have the 4th ed (1997).
>
> One can try reading the Henson/Penny monograph itself, though it's
> relatively heavy going.
|