Hi all,

Excerpts from SPM-help#: 14-Apr-100 Re: Random effects analysis by Rik
Henson@f\
il.ion.ucl.a
> Perhaps I could ask the "community"s feeling on a related
> issue? While in SPM we take con*.imgs through to second-level
> analyses, which are simply linear combinations of the
> parameter estimates, other groups appear to take statistics
> (eg t-statistics) through to second-level analyses
> (equivalent to selecting spmT*.imgs). 


If the interest is in population inference, that is, performing
modeling which accounts for subject-to-subject variability, then the
appropriate approach is the two-level 'RFX' analysis.  If the interest
is simply to combine several fixed effects analyses, answering the
question 'Are all of these individual subject's null hypotheses
true?', then combining p-values or t-statistics in a meta-analytic
fashion would be appropriate.

The random effects model can 'say' more, i.e. makes a statement about
a population parameter.  The meta-analysis approach just makes a
statement about the conjunction of null hypotheses; in particular a
significant result could simply be due to one subject with profoundly
significant data.  The cost of the broader scope of inference is
stronger assumptions.  In particular the first level parameter
estimates are assumed to be

	  1. Normally distributed (across subject), and to have
	  2. Homogeneous variance (across subject).

Point one would be violated, for example, if one subject had a very
large response but all others had a very small response, or, if there
was a bimodal distribution of responses, e.g. half the subjects
responded strongly the other half hardly at all.  Point two would be
similarly violated if a particular subject had a relatively poor
fitting model, or simply if there were dramatic differences in the
variance images across subjects.


> Though the SPM approach may be the
> appropriate test of "effect size", rather than "effect
> significance", it is not clear to me that parameter estimates
> are that meaningful in the context for poor model fits
> (eg when first-level error terms are large). 

Setting aside 'effect' nomenclature, you raise the very important
issue of model fit.  All inference is based on the assumption that you
have your model (nearly) correct.  As pointed out above, the random
effects model has strong assumptions and hence is more susceptible to
departures from assumptions, but I don't find this a convincing reason
to use the meta analysis approach since it is answering such a
dramatically different question.


To issue a slightly different call to the 'community', I find the
conclusion to the above comments to be that model fit is a terribly
important, but oft ignored issue.  Assessment of model fit is a topic
that occupies a large part of a basic linear modeling course, but
one that has been all but ignored in functional neuroimaging.  This is
not to say that it is a trivial issue... model diagnosis for
univariate multiple regression is subjective and practically an art,
but I see it as an area that badly needs work in theoretical and
software domains for fNI. 

-Tom


    -- Thomas Nichols --------------------   Department of Statistics
       http://www.stat.cmu.edu/~nicholst     Carnegie Mellon University
       [log in to unmask]                 5000 Forbes Avenue
    --------------------------------------   Pittsburgh, PA 15213




PS:  I know of only a handful of references that deal with assessing
model fit; I've listed some listed here:


Use of sums of squares images to understand model fit:

Michio Senda, Kenji Ishii, Keiichi Oda, Norihiro Sadato, Ryuta
Kawashima, Motoaki Sugiura, Iwao Kanno, Babak Ardekani, Satoshi
Minoshima, Itaru Tatsumi. "Influence of ANOVA Design and Anatomical
Standardization on Statistical Mapping for PET Activation."
Neuroimage (Oct 1998) 8(3):283-301.


In chapter two there is extensive study of model selection
issues, an inferential approach to model fit:

Holmes, AP. "Statistical Issues in Functional Brain Mapping".}, PhD
Thesis, University of Glasgow. 1994.  (Available from 
http://www.fil.ion.ucl.ac.uk/spm/papers/APH_thesis )


Another approach to model selection using KL distance:

PL Purdon, V Solo, RM Weisskoff, E Brown.  "Model Comparison for fMRI
Data Anlaysis".  Neuroimage (June 1999) 9(6,2/2):S34.




%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%