Dear Thilo,
This is a difficult issue and I don't have a definitive answer, although my intuition would be to have more faith in the separately optimised approach. But ...
Perhaps you can help by describing the models in more detail. Let's say m_opt
is the model chosen by separate multiple optimisations, and m_post is the model chosen by post-hoc selection. In what ways are m_opt and m_post different, in terms of connections ?
As well as using the post-hoc method for approximating the model evidence it is also possible to use it to estimate the connection values - see equation (18) in [1] - I'm wondering in what way the post hoc connections for m_opt differ from the 'optimised' connections for m_opt ...
Sorry this is such a late reply.
Best wishes,
Will.
[1] M. Rosa and K. Friston and W. Penny (2012). Post-hoc selection of dynamic causal models. Journal of Neuroscience Methods Available online 3 May
> -----Original Message-----
> From: SPM (Statistical Parametric Mapping) [mailto:[log in to unmask]]
> On Behalf Of Thilo Kellermann
> Sent: 15 November 2012 10:48
> To: [log in to unmask]
> Subject: [SPM] DCM post-hoc BMS vs. standard BMS
>
> Dear DCMers,
>
> in a recent publication by Friston and Penny (2011, 'Post-hoc Bayesian
> Model Selection' NeuroImage), a new approach of post-hoc model
> selection was suggested. This post-hoc BMS uses a "full model" of all
> possible connections between previously defined regions in order to get
> an optimised model which is expected to be sparser than the full one.
>
> The reason for us to choose post-hoc BMS was the fact that an
> exhaustive search is almost impossible since about 2^60 models would
> have to be inverted for each subject (we had 4 regions and 5 different
> modulatory inputs). Using post-hoc BMS we obtained an optimised
> (reduced) model which was expectedly much less complex than the full
> model and had a posterior close to 1.
>
> However, we had previously tried to approach an optimal model by a
> standard BMS for which we specified hundreds of different models. After
> inversion of all these models and subsequent (standard) BMS, the
> posteriors peaked on one model (posterior probability >0.99).
>
> Having done this work, we were curious if a standard BMS would select
> the optimised model using post-hoc BMS when we included this in the
> model space. After standard BMS (including the optimised DCMs of all
> subjects) the posteriors surprisingly still peaked on the same model
> which had been selected in the previously described standard BMS -
> again with a posterior of >0.99.
>
> (We tried both: on the one hand, we included the "DCM_opt_*.mat" for
> each subject which was automatically created during post-hoc BMS. On
> the other hand we manually defined and inverted DCMs which had the same
> structure compared to the optimised DCMs (ones for all non-zero
> parameters) but this did not affect the results of the BMS.)
>
> The logical question now is obvious (I guess): Which of these BMSs
> should we trust and rely on?
>
> On the one hand one might think that the standard BMS is more
> "accurate" (whatever this means...), because conclusions are based on
> inverted models within the model space. On the other hand one wouldn't
> necessarily expect that a model space comprised of "only" several
> hundreds of models (compared to about 2^60 possible ones) includes a
> DCM which is superior to one based on post-hoc BMS (that quickly
> searches the subspace of a full model).
>
> Any comments are appreciated.
>
> Kind regards,
> Thilo
>
> --
> Thilo Kellermann
> RWTH Aachen University
> Department of Psychiatry, Psychotherapy and Psychosomatics JARA
> Translational Brain Medicine Pauwelsstr. 30
> 52074 Aachen
> Germany
> Tel.: +49 (0)241 / 8089977
> Fax.: +49 (0)241 / 8082401
> E-Mail: [log in to unmask]
|