Dear Ian,

>i continue to wonder
>if it is necessarily a priori biased to somehow weight each
>subject by the quality of their fit when performing the second
>stage t-test during an rfx analysis.  on the one hand its seems
>legitimate since a subject can be poorly fit regardless of the size
>of the height estimate (or difference estimate).  in practice however,
>i would expect that subjects with small estimates would in
>fact have larger residuals.

Here is an interchange from back in April between Rik Henson and 
Thomas Nichols which deals with the issue of the difference between 
taking the t images through and taking the betas through to the 
second level.  Thomas Nichols also touches on the issue of 
within-subjects variance having to be the same from one individual to 
another, but without on this occasions trying to explain why.  Rik 
mentioned this message to me earlier, but it took me a while to find 
it!

Rik asked:
>  > Perhaps I could ask the "community"s feeling on a related
>  > issue? While in SPM we take con*.imgs through to second-level
>>  analyses, which are simply linear combinations of the
>>  parameter estimates, other groups appear to take statistics
>>  (eg t-statistics) through to second-level analyses
>  > (equivalent to selecting spmT*.imgs).
>

Thomas Nichols replied:
>If the interest is in population inference, that is, performing
>modeling which accounts for subject-to-subject variability, then the
>appropriate approach is the two-level 'RFX' analysis.

[And I think that by this he meant taking the beta images through to 
the second level, as we are advised to do in SPM.]

>If the interest
>is simply to combine several fixed effects analyses, answering the
>question 'Are all of these individual subject's null hypotheses
>true?', then combining p-values or t-statistics in a meta-analytic
>fashion would be appropriate.

[Whether this just means taking the t images through to the second 
level and using them exactly as you would have used the beta images I 
am not quite sure.  Perhaps 'meta-analytic fashion' indicates 
something more fancy.  Anyway, he continues...]

>The random effects model can 'say' more, i.e. makes a statement about
>a population parameter.  The meta-analysis approach just makes a
>statement about the conjunction of null hypotheses; in particular a
>significant result could simply be due to one subject with profoundly
>significant data.  The cost of the broader scope of inference is
>stronger assumptions.  In particular the first level parameter
>estimates are assumed to be
>
>	  1. Normally distributed (across subject), and to have
>	  2. Homogeneous variance (across subject).
>
>Point one would be violated, for example, if one subject had a very
>large response but all others had a very small response, or, if there
>was a bimodal distribution of responses, e.g. half the subjects
>responded strongly the other half hardly at all.  Point two would be
>similarly violated if a particular subject had a relatively poor
>fitting model, or simply if there were dramatic differences in the
>variance images across subjects.

Anyway, back to your comments:
>i would guess that the answer to this rests on exactly what one wants to
>say about the "population".  in the extreme such a weighting approach
>would face the same problem as a fixed effects approach,
>in that a minority of the sample could determine the conclusions
>(i.e. really poorly fit subjects would be weighted negligibly).

I think that the situation is worse than this.  If all of the 
subjects are selected randomly for a fixed-effects analysis, then it 
could be thought of as a useful case study for understanding the 
characteristics of the population from which it was drawn.  You don't 
know how typical these subjects are of the population (that's what a 
RFX analysis is for), but there's no particular reason for thinking 
that your results are biased in one particular direction or another. 
However, if you do an analysis which is weighted towards low-variance 
individuals, then it's as though the data itself is being used to 
select the subjects, which seems much more worrying even to a 
non-statistician such as myself.

Let's imagine that, unbeknownst to you, there are two sub-groups in 
your 'population', a large sub-group with quite a bit of variance, 
and a small sub-group with unusually small variance.  These 
sub-groups appear in appropriate proportions in your sample.  A 
random-effects analysis will tend to give a result which is fairly 
representative of the population taken as a whole (in spite of the 
departure from the homogenous-variance assumption which strictly 
speaking renders it invalid, I now learn).  However, taking the t 
images through will bias the results towards the low-variance 
sub-group, which seems a much worse thing to do.

>nevertheless,
>i can't help thinking that taking 15 or 20 gigabytes of subject data,
>reducing it to 12 or so synthetic height estimates that are all considered
>equally
>valid, and then asking whether the 95 percent CI of these
>encompasses zero is a bit extreme.

I am completely in tune with this sentiment.  I really fear the swing 
towards referees demanding random-effects analyses in all fMRI 
studiesin a knee-jerk manner, and the consequences for any department 
in which scanning time is limited.  As pointed out by Karl and 
others, one can ask many useful questions about the brain using 
fixed-effects analyses, and one has to have a very good reason indeed 
to throw away all of the useful information about how reliable your 
observations are within the individual brains in which they were made.

Clearly if you want to compare two groups, then RFX analyses are 
mandatory.  On the other hand, once Hubel and Wiesel had demonstrated 
ocular dominance columns electrophysiologically in several brains, 
one wouldn't have valued the services of a referee who refused to 
allow the paper to be published until they had statistical evidence 
that these observations could be extended to the population!

>one alternative that we have tried is to start with the fixed effects
>analysis, and then followup by asking if the regions identified in the fixed
>effects are reliable across the subjects in the sample using roi extraction
>and random effects anova on the extracted timecourse data for each subject.

Sorry, I didn't quite follow this.  Perhaps someone else will comment.  But...

>if the condition effects and condition by time interactions are significant,
>one can assume the activation is reliable in the sample.

...surely the standard fixed-effects analysis already tells you this?

>the presumption
>then is that given another sample of 7 or 8 right-handed, undergraduate,
>native-english speakers, i should expect a similar outcome.  of course this
>is an reasoned versus statistical "population" inference.  i'd be
>interested if you or anyone has any comments regarding the
>veracity of this method.

I agree wholeheartedly that such a 'reasoned' population inference 
may be entirely appropriate for many questions (see above).

>cheers.
>ian.

I hope that you get some other (better-informed) responses to your 
messages soon.  Sorry that I seem to be incapable of writing a short 
message to this list!

Best wishes,

Richard.


-- 
from: Dr Richard Perry,
Clinical Lecturer, Wellcome Department of Cognitive Neurology, 
Institute of Neurology, Darwin Building, University College London, 
Gower Street, London WC1E 6BT.
Tel: 0207 679 2187;  e mail: [log in to unmask]


%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%