Dear Matthias,

There is no need for any further comparisons since you have already done the right thing, i.e. applying BMS to the entire space of models considered. You can now simply report which model was optimal, state its expected posterior probability (or its exceedance probability) in relation to the other models, and report its parameter estimates.

Generally, note that selective comparisons of subsets of models are not equivalent to a proper model space partitioning approach unless you are using the agglomerative property of the Dirichlet distribution. As explained in the discussion of our BMS paper, the posterior belief about which model is most likely to have generated the data is a function of the entire set of models considered. This means that changing model space (by reducing or extending the number of models considered) can change one's inference about the optimal model. As a consequence, one should always infer the most likely model by comparing the entire set of plausible models at once.

Best wishes,

Klaas

There is no need for any further comparisons since you have already done the right thing, i.e. applying BMS to the entire space of models considered. You can now simply report which model was optimal, state its expected posterior probability (or its exceedance probability) in relation to the other models, and report its parameter estimates.

Generally, note that selective comparisons of subsets of models are not equivalent to a proper model space partitioning approach unless you are using the agglomerative property of the Dirichlet distribution. As explained in the discussion of our BMS paper, the posterior belief about which model is most likely to have generated the data is a function of the entire set of models considered. This means that changing model space (by reducing or extending the number of models considered) can change one's inference about the optimal model. As a consequence, one should always infer the most likely model by comparing the entire set of plausible models at once.

Best wishes,

Klaas

Dear Dr. Stephan,

I'm glad to hear that the model space partitioning approach will be further

developed. At the moment, i would like to write up the study of which i was

showing the results - with 16 models better than 48. Until the new method is

available: do you think a simple comparison of a single model out of the

best 16 (the worst among them) against a single model out of the worst 48

(the best among them) would also show that those 16 models are superior to

the 48?

Please see the .pdf file for clarification.

Matthias