Dear Dav,
you are right that the df is indeed a concern, and inflating them by
putting in more than two contrasts per subject is a problem.
If you have one contrast per subject and you simply use a one-sample t-
test to check for activation within the group you have n-1 df (where n
is no of subjects) and that is fine. Likewise if you have one contrast
per subject and two groups.
Things will also be kosher in the case where you have two contrasts
per subject and do a paired t-test.
The problem starts when you have multiple (>2) contrasts per subject.
Lets say you have four contrasts per subject (typical two-factorial
design) and that you put these into a final-level blocked ANOVA (with
subjects as blocks). You will then have have close to 3n df, which is
clearly inflated and not really kosher.
The reason it is not OK is because these measurements will be
correlated, i.e. the residual error across one subjects four scans are
likely to be similar. This is the same kind of problem as the temporal
autocorrelation at the first level where two measurements close in
time will be correlated and hence will not constitute two
"independent" measurements, but maybe just 1.3 or 1.4 measurements.
This has been solved differently by SPM and FSL, which I think
contribute to the slight confusion about what designs can and cannot
be used at the final level.
in SPM they will estimate the full variance-covariance matrix, i.e.
they will try to calculate just how independent those measurements
really are and then use that to "pre-whiten" the data. You can think
of the pre-whitening as "correcting the df", though in practice they
increase the variance instead (but I think it is harder to get ones
head around that). The process is very similar to the pre-whitening
that is being done at the first level in both SPM and FSL.
FSL has taken the different route to NOT estimate the covariances at
the final level, which means that you as a user will have to make sure
that the design is such that you don't inflate the df (as per the
examples above). One reason that FSL can do this (without losing
power) is because, unlike SPM, FSL passes information both about the
(contrast of) the parameters (the COPE) and the uncertainty of those
parameters (the VARCOPE) from one level to another.
That means that in FSL the recommended way to analyze the two-
factorial experiment I alluded to above is to create the relevant [1
-1 -1 1] contrast for each subject at the first level and then pass
that (along with the uncertainty of that estimate) up to the second
level and there simply do a one sample t-test.
I hope this has answered your question?
Best regards Jesper
On 11 Nov 2009, at 00:48, Dav Clark wrote:
> I hadn't thought about it from a dof perspective, so given that, the
> approach I'm actually using seems to be more conservative (I am
> actually fitting each contrast's highest level in a separate model).
> Seems like I could be committing a grave error with the one-big-
> model approach and at best missing out on a much smaller correction.
>
> Certainly, it is not easy to find anything on the forums, and I
> couldn't find anything on this. The docs essentially never treat the
> issue of multiple contrasts.
>
> Thanks David!
> Dav
>
> On Nov 10, 2009, at 8:29 AM, David V. Smith wrote:
>
>> OK -- I think I see your concern now, but I don't think you really
>> have to worry about that (just dig through the forums a bit more,
>> and I know you'll find a post where this has come up before). As
>> far as I know, this is a fairly standard way of doing the subject-
>> level analysis that isn't just specific to FSL...
>>
>> With your way, it seems like you're giving yourself free DFs (since
>> the number of inputs you'd have would be subjects*copes). I guess
>> that may not be a concern, but it still seems like an odd way build
>> a model for a specific prediction. One of the FSL folks will have
>> to respond with a more detailed information if you can't find what
>> you're looking for on the forums... Sorry.
>>
>> Cheers,
>> David
>>
>>
>>
>> On Nov 10, 2009, at 2:08 AM, Dav Clark wrote:
>>
>>> I'll try a different explanation:
>>>
>>> 1st level: Separate first-level analysis for each run, containing
>>> multiple contrasts - e.g. subject1/scan1.feat
>>>
>>> 2nd level: FE analysis across run, separately for each subject
>>> (group
>>> average) - e.g. subject1/scan1.feat + .../scan2.feat + ... ->
>>> subject1/all_scans.gfeat
>>>
>>> 3rd level: ME analysis across all copes for a given contrast (group
>>> average) - e.g. subject1/all_scans.gfeat/cope1.feat + subject2/<the
>>> same> + etc.
>>>
>>> I understand you could do it a slightly different way and end up
>>> with
>>> cope images instead of cope feat directories. My way seems to work
>>> (I'm fortunate in that I have a simple primary motor activation
>>> that's
>>> guaranteed to come out in one of my contrasts). I'm just concerned
>>> that I'm cheating in terms of family-wise error, etc. The
>>> alternative
>>> I see is incorporating all copes into a single ME analysis, with a
>>> separate contrast for each lower-level contrast (corresponding to a
>>> single EV that picked out all contrasts of a particular type).
>>>
>>> Cheers!
>>> Dav
>>>
>>> On Mon, Nov 9, 2009 at 7:08 PM, David V. Smith <[log in to unmask]
>>> > wrote:
>>>> I'm not sure I completely understand what you're asking. I
>>>> typically do a
>>>> 3rd level analysis for each cope using the cope image
>>>> (subject.gfeat/cope1.feat/stats/cope1.nii.gz) as the input (i.e.,
>>>> one
>>>> contrast at a time). I've never done it the way you suggest --
>>>> under a three
>>>> level set up (runs/subjects/group), I thought we had to go one
>>>> cope at a
>>>> time. The only times I have multiple cope images in a 3rd level
>>>> model is
>>>> when I'm doing ANOVAs or paired t-tests -- never for simple main
>>>> effects.
>>>>
>>>>
>>>>
>>>> On Nov 9, 2009, at 7:04 PM, Dav Clark wrote:
>>>>
>>>>> Yeah - I was pretty sure that was OK.
>>>>>
>>>>> One remaining question, though, one could do a single mixed
>>>>> effects model
>>>>> with an EV for each contrast, and then contrasts picking out
>>>>> each EV
>>>>> separately. OR, you can do a mixed effects model for just
>>>>> contrast 1 (i.e.
>>>>> all cope1.feat directories), then another for each remaining
>>>>> contrast.
>>>>>
>>>>> Does that have any effect on results?
>>>>>
>>>>> Thanks again!
>>>>> DC
>>>>>
>>>>> On Nov 9, 2009, at 3:57 PM, David V. Smith wrote:
>>>>>
>>>>>> I actually do a fixed effects analysis for each subject
>>>>>> individually --
>>>>>> and that produces the output you say say you expect to see. But
>>>>>> as long as
>>>>>> it's FE, it shouldn't make a difference (cf.
>>>>>> https://www.jiscmail.ac.uk/cgi-bin/webadmin?A2=ind0908&L=FSL&P=R474)
>>>>>> .
>>>>>>
>>>>>> The FSL folks will have to look into your request about
>>>>>> clarifying the
>>>>>> documentation here.
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Nov 9, 2009, at 3:22 PM, Dav Clark wrote:
>>>>>>
>>>>>>> On Nov 9, 2009, at 6:19 AM, David V. Smith wrote:
>>>>>>>
>>>>>>>> Alternatively, are you just trying combine multiple sessions
>>>>>>>> that all
>>>>>>>> have the same conditions? If so, the solution is
>>>>>>>> easy:http://www.fmrib.ox.ac.uk/fsl/feat5/detail.html#MultiSessionMultiSubject
>>>>>>>> .
>>>>>>>> You could also do this a bit differently by doing a second
>>>>>>>> level fixed
>>>>>>>> effects analysis for each subject.
>>>>>>>
>>>>>>> Actually, I am also struggling with the docs here right now.
>>>>>>> Specifically, after running an FE analysis as suggested in the
>>>>>>> docs there,
>>>>>>> you are supposed to:
>>>>>>>
>>>>>>> "select the 5 relevant directories created at second-level,
>>>>>>> named
>>>>>>> something like subject_N.gfeat/cope1.feat"
>>>>>>>
>>>>>>> Thus I would expect something like:
>>>>>>>
>>>>>>> subject_1.gfeat/cope1.feat
>>>>>>> subject_2.gfeat/cope1.feat
>>>>>>> ...
>>>>>>>
>>>>>>> But this is not what happens. You instead get a single gfeat
>>>>>>> directory
>>>>>>> (named whatever you said to call it) that contains a cope
>>>>>>> directory for each
>>>>>>> subject. In your example, you'd get something like
>>>>>>>
>>>>>>> fixed_eff.gfeat/cope1.feat
>>>>>>> ...
>>>>>>> fixed.dff.gfeat/cope5.feat
>>>>>>>
>>>>>>> (i.e. a copeN directory corresponding to each subject)
>>>>>>>
>>>>>>> Thus, a reasonable person might assume either the first part
>>>>>>> or the
>>>>>>> second part of these instructions is misleading and assume
>>>>>>> either:
>>>>>>>
>>>>>>> 1) I should do a fixed effect model separately for each
>>>>>>> subject's set of
>>>>>>> runs (thus obtaining subject_N.gfeat directories for each
>>>>>>> subject - 5 such
>>>>>>> in the example above with 3 copeN.feat directories in each).
>>>>>>> Then, I simply
>>>>>>> select the cope1.feat from each subject and do a flame model
>>>>>>> on that, then
>>>>>>> again for the remaining two contrasts.
>>>>>>>
>>>>>>> 2) or perhaps I should do the first part according to the
>>>>>>> instructions
>>>>>>> and then just select those 5 cope directories for each subject
>>>>>>> in the fixed
>>>>>>> effects gfeat directory. (this is what I did)
>>>>>>>
>>>>>>> It's not clear to me if there'd be any difference
>>>>>>> mathematically in the
>>>>>>> above - perhaps some correction for multiple comparisons in
>>>>>>> the latter?
>>>>>>>
>>>>>>> In any case, I think the wording there could be cleaned up
>>>>>>> just a bit
>>>>>>> and it'd make the docs a lot nicer to use.
>>>>>>>
>>>>>>> Cheers,
>>>>>>> Dav
>>>>>>
>>>>
>
|
