Dear Carmen,
obviously you did a random effects (RFX) Bayesian model selection (BMS).
Furthermore, it sounds as if you observed that there were only the same
few models in Occam's window but nevertheless the exceedance probability
of the winning model at the group level is quite low.
I guess the approach you proposed is not valid, but if you assume that
there has to be one DCM which "generated" the data of all subjects, you
can perform a fixed effects (FFX) BMS. This might be a solution for your
problem If you acknowledge the above mentioned assumption. With FFX BMS
Occam's razor is not applied because of that assumption.
The "p-values" are actually the posterior probabilities of the model
given the data (and hence not to be confused with traditional p-values,
reflecting the probability of the data given the null-hypothesis). These
are based on the free-energy of each model (DCM.F) which represents a
lower bound on the log-evidence of the model (something like the
accuracy minus the complexity of the model).
The usage of Occam's razor after RFX BMS is for computational expedience
only and rests upon the fact that models with low posterior
probabilities contribute little to BMA (Penny et al. 2010, PLoS
Computational Biology). Thus, the criteria for Occam's razor should not
be modified, unless you want to include more (or all) models in the BMA.
This is also the reason why Occam's razor is only applied when doing
BMA: it is just a computationally efficient way for averaging the models
within the model space.
Furthermore, Occam's razor requires the posteriors of the models. So
changing the so called minimal posterior odds ratio for in/exclusion
in/from Occam's window has no effect on the posteriors and I guess it
should not have an effect on the exceedance probabilities.
Hope this helps,
Thilo
On Fri, 2013-11-29 at 16:01 +0000, Carmen wrote:
> Dear SPM-experts,
>
> I defined 128 DCM models with each model having the same intrinsic connections but differing direct inputs and modulators. Performing BMS none of the models came up as a clearly winning model; the best model reaches a model exceedance probability of about 0.09. Is it a valid approach to choose for each subject only the models which appear in the Occams window and perform BMS with the models all subjects have in common? According to which criteria are the models chosen to be in Occams window and can I modify these criteria? And why is Occams razor only applied when I do BMA? Are the p values for each model appearing in Occams window calculated before or after using Occams razor?
> Thank you very much for your help,
> carmen
|