 |
 |
Dear Becky, So, to summarise, you have 3 models which I'll call (i) intrinsic connections with correct direction (CD), (ii) intrinsic connections with incorrect direction (ID) and (iii) a union of the two which has both sets of connections (U). Your results are that U wins, even though CD is the correct model. With a 'glass half full' perspective this result is at least reassuring in that ID does not win. Is CD more likely than ID ? One feature we've noticed about model comparison is that it is harder to find in favour of a nested model than its parent when the nested model is true (see eg. scales in Figure 8 of [1]) than it is to find in favour of the parent model when the nested one is true (see eg Figure 7 of [1]). So your results are consistent with this. Best, Will. [1] W. Penny (2011). Comparing Dynamic Causal Models using AIC, BIC and Free Energy. Neuroimage Available online 27 July 2011. > -----Original Message----- > From: SPM (Statistical Parametric Mapping) [mailto:[log in to unmask]] > On Behalf Of Becky van den Honert > Sent: 07 May 2012 18:18 > To: [log in to unmask] > Subject: Re: [SPM] DCM fails to select model that generated the data > > Dear Will (et al) > > I have re-run my simulation taking into account your suggestions, but a > model that is not the true model is still the very clear winner (using > both FFX and RFX). Specifically I’ve done the following: > > Simulated data for 10 subjects (three 5-min runs each) using > spm_dcm_generate.m TR = 0.240 sec TE = .025 sec A = [-1 0 0 0; 0 -1 0 > 0; .1 .9 -1 .1; .9 .1 .1 -1] B = all zeros C = [.8 .1; .2 .8; 0 0; 0 > 0] SNR = 5 transit = [.98 .98 .98 .98] (from Table 1 of Friston et al, > 2003) decay = [.65 .65 .65 .65] epsilon = .05 > > Again, I included just three models in model space. The first was the > one used to generate the data. The second was the “opposite” model in > which the intrinsic connections pointed the other direction and the > direct inputs entered in the opposite pair of regions. The third was > the union of those two models. > > Again, the third model (union model) is the very, very clear winner. > However, during estimation, the EM procedure often had to go through > all 64 steps allowed (and did not always report “convergence”). This > makes me wonder if the EM procedure did not settle on the optimum > estimates for each model. > > Perhaps examining the parameter estimates, as you suggested, will help > clarify this? You said to make sure they were close to the true values, > especially for those different between models. I’ve checked this for > the true model and the winning alternative model, using BPA to get a > quick look at the estimates. A diagram of this is attached (thickness = > magnitude, red = negative, dotted = post.prob < 90%). It seems that > they were not always the same as the true values, but often close. The > hemodynamic parameter estimates are farther from the "truth" and > priors, but they are similar across models. > > Also, I ran Dr. Friston’s spm_dcm_fmri_check.m inputting the > DCM_avg.mat file created using BPA for each model. Both models > explained a very large amount of the variance and yielded no “red- > flags”. > > (By the way, I also followed your recommendation to check that the true > parameters “are within the range of the priors” – which seems unlikely > to be the problem given your recent discussion with Martin. > Nevertheless, they are.) > > Thank you in advance for any guidance or suggestions. I’d be happy to > follow-up with more details. > > Becky