Dear Tali Bitan,
when partitioning the model space, the exceedance probabilities are
calculated in order to obtain exceedances for the families of models -
not for single models. Strictly speaking, when partitioning the model
space one would only refer to the winning family of models (and their
posterior and exceedance probabilities) but not to the values of
individual models (although these values are stored somewhere in the BMS
structure).
Winning models (as well as winning families) are determined according to
their posterior probabilities and not with respect to their exceedance
probabilities. I can only speculate that the posteriors across the model
space do not favor a single model nor a single family of models in your
data. In such cases it is not very surprising that exceedance
probabilities have their maximum on different models when partitioning
is introduced or changed.
Kind regards,
Thilo
On Sun, 2014-07-06 at 01:07 -0400, Tali Bitan wrote:
> Dear Thilo Kellerman
>
>
> Thanks a lot for your reply. This is very helpful to know.
> However, given that all of our families are the same size - what else
> can cause this dramatic change in exceedance probability?
>
>
> Attached are the figures with the xp for our 16 models. Each family
> includes 4 models (1-4, 5-8, 9-12, 13-16). XP for Model 11 which was
> the winning model in the first BMS shrank to zero in the BMS With
> family partitions. It should be noted that none of the winning models
> are included in the winning family (family 2: models 5-8, xp=0.45).
>
>
> Thanks again
> Tali Bitan
>
> ==============
> Tali Bitan, PhD
> Department of Communication Sciences & Disorders
> University of Haifa
>
> Visiting Professor,
> Department of Speech-Language Pathology
> University of Toronto
>
>
>
> On Fri, Jul 4, 2014 at 10:39 AM, Thilo Kellermann
> <[log in to unmask]> wrote:
> Dear Tali,
>
> as far as I know this is not an inconsistency because
> posterior
> probabilities surely depend on prior probabilities - and
> different
> partitioning of your model space likely goes along with
> different prior
> probabilities for single models. This may be easy to see if
> you imagine
> that per default priors are assumed to be flat. Without
> partitioning
> this means that each model within the space has a prior of 1/n
> with n
> being the number of models. When the space is partitioned,
> however, the
> priors are not assumed to be flat with respect to single
> models but with
> respect to the partitions. Consequently, the prior of a single
> model
> depends on the size of the partition it belongs to with lower
> priors for
> models in large partitions and higher priors for models in
> small
> partitions.
>
> Hope this helps,
> Thilo
>
> On Fri, 2014-07-04 at 06:40 +0100, Tali Bitan wrote:
> > Dear DCM experts
> >
> > While running RFX BMS analysis we have noticed some
> inconsistency.
> > The exceedance probability for individual model changes when
> adding a family partitioning comparison to the process.
> > Namely: when we run BMS on 16 models without family
> partitioning - model 11 is the winning model. However, when
> running BMS with the same data, while also testing for family
> partioning - suddenly model 14 is the winning model.
> >
> > My question is: why does family comparison affect the
> exceedance probability of individual models?
> >
> > I'd appreciate your help
> > Tali Bitan
>
>
>
>
|