Hi Zsuzsi,

I take that you meant to send this to the list, so I'm forwarding. Please, see below:

On 22 February 2015 at 11:40, Z. Sjoerds <[log in to unmask]> wrote:

Hi Anderson!

Thanks so much for your help! It seems a struggle to me, to fully comprehend why you make certain choices for a design matrix. But I do want to fully comprehend, si I am sure I do the right thing in the future.

An issue that is left for me, is what you say in the beginning of your last email: all EVs that are set to 0 are nuisance variables. This is what I know also from the SPM design matrices. Statistically this means that the variance explained by these variables is left out ('regressed out') in your test. But when I compare the two groups on FA value (independent of behavioral outcome; in principle a whole different question), I don't want any variance explained by the behavioral measurements to be taken out. I might miss some important group differences due to the fact that one of the behavioral measurements differs significantly between groups. At this level of hypothesizing, I don't want anything to do with the behavioral measurements (yet). Only in the later step, when I regress FA value with behavior, I could isolate a possible group*behav effect.

With 50 subjects, these 4 EVs will hardly become an issue, so I don't think there's a need for multiple models. But if you note that the EVs with the behavioural variable are not significant, and if it isn't anyway a sensible hypothesis or evidence that they'd somehow be associated with FA, you can remove them from the model and run just a 2 group comparison (presumably with age, as you mentioned).

 

So therefore I thought that I need two design matrices: one for the simple FA group comparison:

Group = group (25 ones, 25 twos)

EV1 = group1 (25 ones, 25 zeros)

EV2 = group2 (25 zeros, 25 ones)

EV3 = nuisance variable (age, demeaned)

Then the contrasts of interest would be:

C1 = F-contrast: 1 -1 (main effect of group)

C2 = T-contrast: 1 -1 (post-hoc group1 > group2)

C3 = T-contrast: -1 1 (post-hoc group2 > group1)

This way I actually regress out the only nuisance variable of interest (and no additional variables that are not nuisance to me)

Then I will run the Two-Sample t-test:

randomise -i AllFAmerged4D -o TwoSampT -d design.mat -t design.com -mask FAthresholdMask -x --T2

There's no need for the F-test, although if you use randomise as shown, it won't make any difference because you didn't use the "-f design.fts". If the reason for the F-test is to account for multiple testing, you can use Bonferroni (0.05/2), It won't be conservative in this case.

Also, variables that aren't in the contrast are nuisance, regardless of them being in fact interesting in some other contrasts (or in any other way) or not. And in general, if a variable may affect the response variable (or is known to affect), it can be kept, even if not significant, as it absorbs some of the variance and makes the test for the other variables more powerful.

 

(Importantly: The 4D file is ordered according to participant number, not according to group. But ofcourse in my design I kept the same order, so e.g. for the first columns group: 1 2 1 1 2 2 1 2 2 1 1 1 2 2 1 2 2 1 1 2 etc.. and the same order for EV1 and EV2; I assume that is fine?)

The order of the rows don't matter as long as they match the 4D file.

 

Second, I will make a design matrix for the second, behavioral-related question (and then defining both behav measurements in one design):

Group = group (25 ones, 25 twos)

EV1 = group 1 * behav 1
EV2 = group 2 * behav 1
EV3 = group 1 * behav 2
EV4 = group 2 * behav 2
EV5 = nuisance variable (age, demeaned)

***So, do I need an intercept here???*** or do I need to define the two groups by two extra EVs here anyways?

Must include the groups.

 

And then the contrasts of interest:

C1 = T-contrast: 1 1 0 0 0 (main positive effect of behav1, independent of group)

C2 = T-contrast: -1 -1 0 0 0 (main negative effect of behav1, independent of group)

C3 = T-contrast: 0 0 1 1 0 (main positive effect of behav2, independent of group)

C4 = T-contrast: 0 0 -1 -1 0 (main negative effect of behav2, independent of group)

C5 = F-contrast: 1 -1 0 0 0 (group interaction test on behav1)

C6 = T-contrast: 1 -1 0 0 0 (post-hoc t-test group1*behav1 > group2 *behav1)
C7 = T-contrast: -1 1 0 0 0 (post-hoc t-test group1*behav1 < group2 *behav1)

C8 = F-contrast: 1 -1 0 0 0 (group interaction test on behav2)
C9 = T-contrast: 1 -1 0 0 0 (post-hoc t-test group1*behav2 > group2 *behav2)
C10 = T-contrast: -1 1 0 0 0 (post-hoc t-test group1*behav2 < group2 *behav2)

Is this correct??

I think this was answered in an earlier email, just check that...

No need for these F-tests, just use Bonferroni.

 

Sorry if I keep on repeating myself, but I am afraid I have been doing things wrong until now, and want to make 1000% sure that I do it correctly now, and also understand it independently, so I don't have to bother you in the future anymore ;-)

By the way, the wiki for Glm / Randomise is offline from the fmrib site. Is there a reason for that? I tried to find some answers there, but that didn't work out..

It should be back now.

All the best,

Anderson

 

Thanks again; hope you can help comprehend the last issues!

Best,

Zsuzsi

On Sat, Feb 21, 2015 at 6:21 PM, Anderson M. Winkler <[log in to unmask]> wrote:

Hi Zsuzsi,

Please see below:

On 21 February 2015 at 14:56, Z. Sjoerds <[log in to unmask]> wrote:

Hi Anderson,

Thanks for your quick reply!

I am slightly surprised that I can put all EVs and contrasts in one design; always understood to make different designs for different questions, especially when it involves two different statistical tests: 2-sample t-test for FA comparison between groups, versus multiple regression for group comparisons on slope with a continuous variable (e.g. behavioral measure). 

It's all the same -- all particular cases of the same model.

 

So, how does that work then, for the fact that age is nuisance here, but the behav EVs are not, especially in the case of 'simple' t-test group comparison of FA, irrespective of behavioral measurement?

Each contrast is tested separately, and for each contrast, the EVs marked as 0 are nuisance. The others (non-zero) are effects of interest.

 

Isn't there then also somehow a correction for the behav scores, as is for age, and since the groups differ in one of the behav scores, don't I then remove an important part of the explained variance between the groups (in the simple FA 2-sample t-test)?

Given behav1 and behav2 are correlated, it's impossible to disambiguate them, and as coded, each test with check the unique contribution of the respective scores. It may not be what you want, hence the suggestion to collapse both scores with PCA. I think you mentioned that the correlation is high, and in this case, another possibility is simply ignore one of them (that is, remove either behav 1 or behav 2 it from the design altogether).

 

And I assume with C5 and C6 you don't mean an interaction? Because my first, main question regards the simple FA comparison between groups. The whole-sample (two groups pooled) regression with the behavs and group*behav interaction are part of my second question.

Yes, sorry, the C5 and C6 are not for interactions, just group differences:

C5: [1 -1 0 0 0 0 0]: (group 1 > group 2)
C6: [-1 1 0 0 0 0 0]: (group 1 < group 2)

 

Shouldn't there also be contrasts that look at whole-sample regression with behav? e.g. 0 0 1 1 0 0  (positive association behav 1), 0 0 -1 -1 0 0 (negative association with behav 1), 0 0 0 0 1 1 (positive association with behav 2), 0 0 0 0 -1 -1 (negative association with behav 2)?

If you want, yes, but if the interaction is significant, these contrasts won't be useful. And if the interaction isn't significant, a single EV for behav (not split between groups) is more appropriate.

 

And I assume demeaning should be done based on the whole-sample mean? Not the mean per group? I keep on getting confused on that..

Yes.

 

Last question: I see you did not add the intercept in the design (e.g. EV1 all 1's); so, when exactly do you have to add it (and how does it or does it not influence the need for demeaning)? I did add it in my design before; is that a flaw? Did this likely influence my results? Is there documentation on the role of the intercept here, and when to use it, and when not, so I can also decide in future studies how to set up my design matrix?

The intercept codes the overall mean. Here you want to compare the means of each group, so the intercept is split across both groups, which are tested with C5 and C6.
The intercept can be modelled as a single EV, but this entails modifying other EVs.

All the best,

Anderson

 

On Sat, Feb 21, 2015 at 10:45 AM, Anderson M. Winkler <[log in to unmask]> wrote:

Hi Zsuzsi,

For these hypotheses, you need in the design matrix:

EV1: group 1
EV2: group 2
EV3: group 1 * behav 1
EV4: group 2 * behav 1
EV5: group 1 * behav 2
EV6: group 2 * behav 2
EV7: age

Mean-center behav 1 and behav 2 before computing the interaction EVs.

The contrasts will be:
C1: [0 0 1 -1 0 0 0]: interaction group vs behav 1
C2: [0 0 -1 1 0 0 0]: interaction group vs behav 1 (opposite sign)
C3: [0 0 0 0 1 -1 0]: interaction group vs behav 2
C4: [0 0 0 0 -1 1 0]: interaction group vs behav 2 (opposite sign)
C5: [1 -1 0 0 0 0 0]: interaction group vs behav 1 (group 1 > group 2)
C6: [-1 1 0 0 0 0 0]: interaction group vs behav 1 (group 1 < group 2)

All these hypotheses can be done with t-tests, so no need for F-tests. The 6 contrasts mean multiple testing. You can use Bonferroni over the 6 tests, so the significance level becomes 0.05/6 = 0.00833. It's not as conservative as it may appear, because the positive and negative versions can be treated as independent with data as you have.

Still, if you want, you can run an F-test comprising C1, C3 and C5. The result of this test is just a shield to minimise multiple testing, as the interpretability is a bit difficult given that the respective t-tests test entirely different things.

The above ignores the fact that behav 1 and behav 2 are correlated. Maybe you can consider using PCA to derive just 1 new behavioural score that captures the variance of these two.

All the best,

Anderson

On 20 February 2015 at 12:11, Zsuzsika Sjoerds <[log in to unmask]> wrote:

Dear Anderson & FSL list,

Following recent posts on the design matrix for e.g. randomise, I started doubting about my own glm approach. I am originally trained with the SPM gui, where one can build contrasts on the go (e.g. define post-hoc t-contrasts only after seeing an interaction effect in F-contrast). For the FSL/glm approach, all contrasts need to be built beforehand (also post-hoc t-contrasts) to run at once in randomise, so an extra amount of forward-planning is needed.
I read about several approaches that I had not applied so far. I hope someone could help figure out what the best GLM design would be for my analyses.

I have a sample of 50 participants, split in two even groups (group1, group2). First I want to compare the groups on FA values, as obtained by TBSS, and second I want to regress the FA values (whole sample & group interaction) with two behavioral measurements (behav1, behav2). These two behavioral measurements negatively correlate with each other, and both also correlate with age (one shows a positive, the other shows a negative correlation). Therefore I want to add age as a nuisance variable, to make sure I don't look at age effects.

For my study I have the following 5 research questions:
- group difference on FA, irrespective of behavioral measurements
- (whole sample) association with behav1
- group * behav1 interaction (and post-hoc t-test, if the interaction is significant)
- (whole sample) association with behav2
- group * behav2 interaction (and post-hoc t-test, if the interaction is significant)

eventually I want to apply this same approach on tractography data, so it would be good to have things straight on a correct design at this point.

Until now I have created 5 separate design.mat files for the 5 questions above (mainly due to the order in which I explored my dataset and tried out designs in the beginning), but I can imagine this is not optimal due to multiple comparisons, and degrees of freedom?


Therefore my first question: can I create an optimal glm design that combines (several of) these questions?

For instance 3 designs:
1. main group comparison, irrespective of behavioral measurement
2. behav1, with EVs and contrasts exploring both a main effect of behav1, and a group interaction
3. behav2, with EVs and contrasts exploring both a main effect of behav1, and a group interaction

However, in a recent post I read that the main group difference (irrespective of behavioral measurement) and the behavioral regression could be entered in one design, together with the nuisance variable (although now behavior * group interaction was studied in this post, as far as I could see)
 (https://www.jiscmail.ac.uk/cgi-bin/webadmin?A2=ind1502&L=FSL&F=&S=&X=0911D30C4E89A0952E&Y=sjoerds.zs%40gmail.com&P=234382).
So then for me this would look like:
(EV1 = intercept? 50 ones??)
EV2 = group1 (25 ones, 25 zeros)
EV3 = group2 (25 zeros, 25 ones)
EV4 = nuisance (50 demeaned values based on whole sample mean)
EV5 = group1 behav (25 demeaned values - based on whole sample mean? 25 zeros for group2)
EV6 = group2 behav (25 demeaned values - based on whole sample mean? 25 zeros for group1)

In that case it was suggested to simply make the F-contrast 0 1 -1 0 0 0 for group differences, and two t-contrasts for possible post-hoc tests:
0 1 -1 0 0 0
0 -1 1 0 0 0

And then I assume, the whole-sample regression with behav would be contrasted as:

t-contrast 0 0 0 0 1 1 (for a positive association)
and
t-contrast 0 0 0 0 -1 -1 (for a positive association)

and group-interaction on behav would be:
f-contrast 0 0 0 0 1 -1 (plus respective t-contrasts if f-contrast shows significance)

However, not taking the two behav EVs (of interest) into account in the first contrast (0 1 -1 0 0 0), I assume that these behav EVs are also considered nuisance variables, and therefore they influence the explained variance between the groups? This is the reason why I earlier built separate GLMs for the non-behavior related group differences, versus regression with behavior. But do I understand correctly now that it is also fine to combine these EVs in one design? How does randomise see the difference between nuisance variables and variables of interest then?
I do assume that the regressions with the two different behavioral measurements should however be defined in two separate designs, especially because of their colinearity. But if I combine pure (non-behavior related) group difference EVs with behavioral EVs, I have the same first contrast (0 1 -1 0 0 0) in both designs (for behav1 and behav2).. hence my confusion.


One other important question: I have manually demeaned both my behav and nuisance (age) parameters. Therefore I don't add the -D in the command in randomise, but did add an intercept as first EV (filed with ones, and all following EVs move a number). But now I read that it was adviced against it?
So, in my case: do I need to add an intercept, and how do I handle demeaning??

Thanks in advance!
Best,
Zsuzsi

--
Z. Sjoerds, PhD
Postdoctoral researcher

Max Planck Institute for Human Cognitive and Brain Sciences
Fellow-Group Cognitive and Affective Control of Behavioral Adaptation
Group Schlagenhauf, Room C211
Stephanstraβe 1A
04103 Leipzig
Germany

[T]: +49 (0) 341 9940 2471
[F]: +49 (0) 341 9940 2499
[E]: [log in to unmask] / [log in to unmask]

--

Z. Sjoerds, PhD

Postdoctoral researcher

Max Planck Institute for Human Cognitive and Brain Sciences

Fellow-Group Cognitive and Affective Control of Behavioral Adaptation

Group Schlagenhauf, Room C211

Stephanstraβe 1A
04103 Leipzig

Germany

[T]: +49 (0) 341 9940 2471

[F]: +49 (0) 341 9940 2499

[E]: [log in to unmask] / [log in to unmask]

--

Z. Sjoerds, PhD

Postdoctoral researcher

Max Planck Institute for Human Cognitive and Brain Sciences

Fellow-Group Cognitive and Affective Control of Behavioral Adaptation

Group Schlagenhauf, Room C211

Stephanstraβe 1A
04103 Leipzig

Germany

[T]: +49 (0) 341 9940 2471

[F]: +49 (0) 341 9940 2499

[E]: [log in to unmask] / [log in to unmask]