Dear Fuyu
Exceedance probabilities pertain specifically to the random effects model-of-models. There's no "rule", but typically people look for an XP > 0.95 to decide that one model is better than all others in the comparison. However, more often than not, there is no clear winning model (as question 2 suggests you have found). In this case, it may be simpler to report the posterior probabilities of the models rather than the exceedance probabilities. Also, when there's no single winning model, Bayesian Model Averaging can be helpful for summarising the estimated parameters across all models.
For your future analyses, you may wish to consider trying the recently developed PEB framework, which doesn't involve exceedance probabilities - https://en.wikibooks.org/wiki/SPM/Parametric_Empirical_Bayes_(PEB) .
Best
Peter
-----Original Message-----
From: #KWOK FU YU# [mailto:[log in to unmask]]
Sent: 30 May 2017 13:51
To: Zeidman, Peter <[log in to unmask]>; Friston, Karl <[log in to unmask]>
Cc: [log in to unmask]
Subject: DCM winning model
Dear Dr Friston and Dr Zeidman,
Hope this email finds you well. I've some questions with regards to selecting a winning model. It'd be great if I can get any advice.
1. Is there an acceptable exeedance probability before we consider a model the winning model? or as long as it's the highest among all the model?
2. If there's 2 models that are very close in terms of xp (i.e. Model 1, xp=0.24 vs Model 2, xp=0.26), do we still select Model 2 as the winning model? or is there a "rule" that the winning model must be a certain amount higher (in terms of exeedance probability) than other models?
Thanks so much for you time and advice.
Kind Regards,
Fuyu
|