the recent discussion on this phenomenon has been enlightening (to me at
least, as I had not appreciated how easily this could happen).
I have added an extra FAQ on
which shows Dbar can easily be made to increase by fitting a covariate
with no explanatory power.
If Dbar measures 'lack of fit', why can it increase when I add a
Suppose Yi is assumed to be N(0,1) under model 1, and consider a
covariate xi with mean 0 and which is uncorrelated with Y. Then it is
straightforward to show that fitting a more complex model 2: Yi ~ N(b
xi,1) leads to Dbar increasing by 1. The crucial idea is that Dbar
should perhaps not really be considered a measure of fit (in spite of
the title of Spiegelhalter et al (2002)!). Fit might better be measured
by Dhat. As emphasised by van der Linde (2005)
(also available from here
Dbar is more a measure of model 'adequacy', and already incorporates a
degree of penalty for complexity.
Joachim Büschken wrote:
> Hello Bugs List:
> recently, I posted this email:
> I am running a hierarchical logistic regression in Winbugs and have
> this problem: When I add a specific explanatory variable to the
> model, the deviance more than doubles, which implies that adding this
> variable reduces the fit of the model. This does not happen when I run
> the same model (without the random effect, i.e. "non-hierarchical") in
> I received a number of very helpful comments (many thanks to all who
> responded!) which I copied below:
> "In your case, I wonder if the explanatory variable explains the
> hierarchy? I don't know the specifics of your situation, but let's
> say your random effect was gender and your explanatory variable was
> hair length. There might be very good correspondence between gender
> and hair length such that you do not require both in the model. One
> or the other would be sufficient. Might this describe your case?"
> This may happen. I checked by running a correlation between my theta
> vector (random effect) and the explanatory variable. The correlation
> is 0.11 and insignificant. Thus, I do not think that this is the
> problem here. However, how do we prevent this from happening (adding a
> random effect which correlates with an explanatory variable in the model)?
> "David Lindely in a discussion (JRSSB) of Aitkens' paper showed that
> the posterior mean of Log L can have very bad behavior because it
> counts the data twice: once in log L and once in the posterior
> Not being a statistician, I cannot really comment on that.
> "One problem is that adding a variable also increases the size of your
> parameter space a lot in HB models. If you had the same prior
> variances for each beta for the two models, then the prior in the
> second model is much flatter/smaller than the prior in the first
> model. In fact, the rate of "flattness" decreases exponentialy with #
> of dimensions. In a situation were there are a large number of
> parameters to observations, the flattening of the "hat" will lead to a
> flattening of the posterior distribution, which means small values of
> the logL get relatively greater weight in the deviance. So your
> E(LogL) can decrease even though the additional covariate is
> good. Model selection using deviance, or even better, Bayes factors or
> posterior probs of models, can get tricky for large model spaces and
> flat likelhoods. In addition to being hard to compute, seemingly
> small changes in the prior can have a big impact on these measures.
> Asymptotically where the likelihood dominates the prior, everything
> works well.
> I find this last point rather troubling. If this can happen, how do we
> evaluate the fit of a hierarchical model if LogL is "bad"?
> Dr. Joachim Büschken,
> Professor of Marketing
> Catholic University of Eichstätt-Ingolstadt, Germany
> [log in to unmask] <mailto:[log in to unmask]> or [log in to unmask]
> <mailto:[log in to unmask]>
> Phone: (49) 841.937.1976
> Fax: (49) 841.937.2976
> This list is for discussion of modelling issues and the BUGS software.
> For help with crashes and error messages, first mail
> [log in to unmask] To mail the BUGS list, mail to
> [log in to unmask] Before mailing, please check the archive at
> www.jiscmail.ac.uk/lists/bugs.html Please do not mail attachments to
> the list. To leave the BUGS list, send LEAVE BUGS to
> [log in to unmask] If this fails, mail
> [log in to unmask], NOT the whole list
This list is for discussion of modelling issues and the BUGS software.
For help with crashes and error messages, first mail [log in to unmask]
To mail the BUGS list, mail to [log in to unmask]
Before mailing, please check the archive at www.jiscmail.ac.uk/lists/bugs.html
Please do not mail attachments to the list.
To leave the BUGS list, send LEAVE BUGS to [log in to unmask]
If this fails, mail [log in to unmask], NOT the whole list