Print

Print

Dear Arkan

1-      Is the above script correct? How can I mean-corrected before calculating the interaction? What do you mean by saying, "Make sure A and B are mean-corrected before calculating the interaction"? For interaction, I have just 1s and -1s!

 

Yes, your script looks good. When calculating the interaction, you need each regressor you’re multiplying to have mean zero (otherwise the resulting vector will be the interaction plus a constant). In your case, there’s no problem – regressors A and B already have zero mean. If they didn’t, you’d manually mean correct using:

 

A = A – mean(A);

B = B – mean(B);

 

2-      In BMR_all figure (see attached, please), there is a reduced MAP. Is this reduced model related to the Mean_constant covariate?


If I remember correctly, this bar chart has all the parameters in the PEB. There will be one parameter for each of the 4 covariates (constant, A, B, A x B) on each connection. To view the results more clearly, run:

 

spm_dcm_peb_review(BMA,DCM)

 

3-      In BMR_all figure, what is the model space explain? How can I read it?

The search algorithm tried hundreds of a combinations of switching off parameters from the PEB model. The model space shows the models from the final iteration of the search – i.e. the best models it found. The rows are connections which varied between the models and the columns are the models. White means a connection switched on, and black means a connection switched off. (Unfortunately the y-axis labels haven’t displayed nicely there because there are too many. If you want the correct labels for the rows, see BMA.Kname.)

The connection strengths from these 256 models are averaged (weighted by their model evidence) to give the Bayesian Model Average BMA, which is what’s returned in your script.

4-      In BMC figure (see attached, please), what are parameters bars explain for example in covariate 1, and what is the meaning of the effective size? If I would like to draw this figure again, where can I find these values?

These are the connection strengths for the commonalities and the first two between groups differences. It gives you a sense of how many connections were ‘switched off’ due to the search over models. I recommend you don’t use this figure, but instead use the simpler GUI tool spm_dcm_peb_review(BMA,DCM), mentioned above.

5-      In PEB_full_ANOVA figure (see attached, please), where can I find the data of gray parameters and the red error bars for covariate 1 (for example)? 6- In PEB_full_ANOVA figure, how can I explain the main effect of factor A, B, and the interaction between them? If I understand correct, there is a particular model for each regressor! How can I find the direction of the A main effect, for example, this model explain A1>A2 or A1<A2?

I think you forgot to send this figure?

7- I tried Leave-one-out cross validation by design the M matrix with ANOVA style:

    M = struct();

    M.alpha = 1;

    M.beta  = 16;

    M.hE    = 0;

    M.hC    = 1/16;

    M.Q     = 'single';

    Mean_constant=ones(80,1);

    A=[ones(40,1); ones(40,1)*-1]; % 1s A1, -1s A2

    B=[ones(20,1); ones(20,1)*-1; ones(20,1); ones(20,1)*-1];  % 1s B1, -1s B2

   A_B=A.*B;            % Interaction A*B

   M.X = [Mean_constant,A,B,A_B];

  field = {'A'};

   [qE,qC,Q]= spm_dcm_loo(GCM,M,field); % GCM: one model (full model) for each subject

 

Should I run LOOCV with full models for each subject (as I did) or with a particular model for each covariate (output of BMA)? Any hints on how to do that would be very helpful. Now, I think the LOOCV calculated just for the main effect of factor A, is that correct? Could you look to the LOO_cross_validation figure and tell how should read and explain figures, please?

 

The purpose of the cross validation is to ask ‘now that I have found evidence for certain group-level effects in my PEB model, would my effect size be large enough to predict the effect in a new subject?’. This could be used to predict a diagnostic score or category for a patient.

 

You indeed ran this correctly, on each subject’s full DCM model. As you correctly noticed, it will just use the first between-groups difference for the prediction  (and all other covariates are still in the model, so you were correct to include them).

 

The key thing to look at is the top right – the out-of-samples correlation and p-value. That’s the correlation between the predicted covariate (e.g. effect A) and the real value for that subject. Clearly that’s not significant here. The plot underneath shows the value in the design matrix (1 or -1) and the estimated value, for each subject. The real-valued covariate predictions are shown at the top left in the solid line, with the dot line being the ‘true’ value. Underneath is the probability for each subject of having a -1 or 1 in their design matrix (which is useful for diagnostic purposes, if -1 or 1 were patient and control).

 

So the conclusion here is that you cannot confidently distinguish the two levels of effect A (A1, A2) from each other in a previously unseen subject, based on their connection strengths. Note that there is a big dilution of evidence effect-  you should only include connections which you have previously identified as having an group-level effect. To do this, just change the line in your code related to field, e.g.

 

field = {‘A(1,2)’,’A(3,4)’};

 

Change that to include the connections showing an effect, which you identified using spm_dcm_peb_review.

 

Best

Peter

 

Thank you in advance

 

Best,

Arkan

From: Zeidman, Peter <[log in to unmask]>
Sent: 06 June 2017 09:45
To: Arkan Al-Zubaidi
Cc: [log in to unmask]
Subject: RE: PEB

 

Dear Arkan

These are all good questions. I hope you don’t mind me CC’ing the SPM mailing list so others can benefit.

 

I want to study differences in fMRI resting-state effective connectivity between four conditions for 20 healthy subjects. Each subject scanned four times i.e., four conditions (factor A with two level {A1,A2}, and factor B with two level {B1,B2}). To estimate the effective connectivity between 5 ROIs, I used spectral DCM per subject/condition. I test multi-model of connections, and I selected the best model using BMS. The full model for each condition or cross conditions wins. Then I looked at the difference in effective connectivity between conditions by using two-way ANOVA repeated measurements. I did not found any significant results after corrected for multiple comparisons. Then I estimated a second level PEB, and I have some fundamental questions:

 

1-        What is the difference between ANOVA (SPSS) and PEB with ANOVA design matrix?

 

2-        I don’t understand way PEB could be better than classical test (t-test, ANOVA, … etc.), could you give a brief explanation on that direction, please?

 

DCM is a Bayesian or probabilistic framework. This means that the connection parameters have prior and posterior probability distributions. Specifically, the parameters have a multivariate normal distribution, so each parameter is described by its mean (expected value) and covariance (uncertainty).

 

When you use ANOVA with SPSS, you are exporting only the expected value of each connection parameter from DCM. You are therefore ignoring the uncertainty or covariance of the parameters. The PEB framework, by contrast, takes the covariance into account. The PEB framework also brings with it all the advantages of Bayesian statistics, as opposed to frequentist statistics used in SPSS. This means, for instance, you can find evidence for the null (of no effect). You can read more about Bayesian stats in any textbook.

 

Finally, PEB is a hierarchical model. That means it gives you updated subject-specific parameter estimates, finessed after seeing the whole group (i.e. subject-specific posteriors with empirical priors from the group).

 

3- Can I use PEB beta values to examine whether effective parameters have significantly nonzero values instead of the t-test, also should I correct for multiple comparisons (because it is GLM analysis)? In attached (Fig.1), you find PEB for one condition. How can I calculate the p values from beta values or in which parameters should I look in PEB output file for reporting the significant connections? For example, PEB returns me the following parameters

   % PEB.Ep    - posterior expectation of second level parameters. Can I say this is the mean of each parameter?

    %PEB.Cp - posterior covariance of second level parameters. Can I say this is the variance of each parameter?

 

In Bayesian statistics, there are no p-values. Instead, we perform model comparisons and test the probability of one model versus others.

 

Start with the full models for each subject (forget the BMS now) and enter them into the PEB with your ANOVA-style design matrix. Note that the software likes the first column to be a column of ones – i.e. the mean. Now try performing an automatic search over nested PEB models, where some covariates / connections are switched off. The software will take the best models found during the search, and average them (a Bayesian Model Average, BMA). You’ll find that any parameters which didn’t contribute to the model evidence will have been pruned away. For each connection, you will have a posterior probability (BMA.Pp). This is the probability that models with the particular connection switched on were better than models with it switched off.

 

Instructions for this procedure are at https://en.wikibooks.org/wiki/SPM/Parametric_Empirical_Bayes_(PEB) .

 

4- When I used  [PEB,DCM ]    = spm_dcm_peb(GCM,M,field);  SPM returns me DCM connections, and these connections different from DCM first level (I estimated spectral DCM parameters per condition for each subject separately). What spm_dcm_peb calculated here? Why spm_dcm_peb estimate again the first level DCM? If I would like to correlate DCM connections with hormones data, should I use DCM parameters from spm_dcm_peb?

 

These are the subject-specific connection strengths you would have got, if the group-level connection strengths had been entered as the priors for each subject. In other words, they are your updated belief about each subject’s connection after seeing the group. Yes, you should use these connection estimates for any further correlations outside the PEB framework.

 

5- I tried to design 2x2 model at the group level with the fallowing regressors in the PEB design matrix. I am using MATLAB script

1.      Mean

2.      Effect of A (1s for A1, -1s for A2)

3.      Effect of B (1s for B1, -1s for B2)

4.      Interaction between A and B (element-wise multiplication of the A and B regressors)

In this model, I did not take in the account the subject effects. Do you recommend to adding regressors for each subject? And how can I report significant beta values? See Fig.2, please.

 

That’s perfect. (Make sure A and B are mean-corrected before calculating the interaction.) No, you probably don’t need to model subject effects. Being Bayesian, there’s no concept of significance now – there’s just probabilities. Typically, people focus their discussion on parameters with at least 90% or 95% posterior probability – see above.

 

6- I tried to throw away any perimeters do not contribute to the model evidence by using PEB or post-hoc for one condition. I ran a full model on one condition to select the best connections by using both approaches. In the Fig. 3 you see the results. Using post_hoc on the full model not helpful for this condition, you see (Fig 3) that the spm_dcm_peb_bmc approach gave different results for the same data. Do you have any explanation? How can I report and explain the results of spm_dcm_peb_bmc?

 

The old post-hoc approach did Bayesian Model Reduction (BMR) at the first level, on each subject’s DCM. The new approach is to do BMR on the group-level PEB model, which properly accounts for between-subject variability. I wouldn’t expect the same results from each approach. I recommend you use the latter.

 

Feel free to ask if you have any further questions, and do CC the mailing list.

 

Best

Peter