Given the recent discussion of problems with using the inverse gamma
prior for a variance, I thought I would repost a message on this subject
that I originally submitted in April 2001. Some may not have seen that
message. The problem I encountered has to do with likelihoods for
variances that peak at zero. This seems to also be the problem with the
inverse gamma prior in the example from Appendix C of Andrew Gelman's
book, though the example there uses different parameter values from what
I tried (InvGamma(1,1) there versus InvGamma(.001,.001) that I tried).
Message from April 2001:
David Spiegelhalter's discussion calls to mind something I ran into when
I first
tried BUGS, that I thought my be useful to relate to others. I used BUGS
on a
model of the following form:
y(i) = x(i)'*b + u(i) + e(i)
where y(i) are U.S. Current Population Survey estimates of state poverty
rates
for some age group and some year, x(i) is a vector of regression
variables with
corresponding parameter vector b, u(i) is a model error assumed i.i.d.
N(0,tau.u), and the e(i) are sampling errors in the survey estimates
y(i). The
e(i) are assumed independent N(0,tau.e(i)), with the tau.e(i) assumed
known.
(They are actually the reciprocals of smoothed survey estimates of
variances of
the e(i).)
Maximum likelihood estimation of s2u = Var(u(i)) = 1/tau.u in this model
often
results in an estimate of zero because the likelihood function peaks at
zero. A
Bayesian approach with a flat prior for s2u avoids this problem and
produces much
more sensible results. I originally implemented this within Splus by
doing
numerical integration over the marginal posterior of s2u. When I first
tried
BUGS, which does not allow a flat prior on (0,infinity), I used the
dgamma(.001,.001) prior for tau.u, which I saw in some of the examples.
The
result was a posterior for s2u that was concentrated near zero. The
reason is
that the implied prior for s2u = 1/tau.u (as well as the prior for
tau.u) is
concentrated near zero. When such a prior is used with a likelihood that
peaks at
zero, the result is a posterior concentrated at zero. I then switched to
specifying a uniform prior for s2u over (0,M), where M was chosen large
enough so
that the likelihood was negligible beyond M. This produces results that
essentially matched those I got from numerical integration, as it should
have.
The moral of this story is that if the likelihood for a variance
component peaks
at zero, then a prior like dgamma(.001,.001) for the corresponding
precision is
not non-informative. Such a prior is non-informative only if the
likelihood
function gives essentially no weight to values of the variance near
zero. Another
lesson learned is not to simply adopt priors from the BUGS examples (or
any other
examples you find) without thinking about whether they make sense for
analysis of
your data.
--
William Bell
SRD, Room 3000-4, Mail Stop 91
U.S. Bureau of the Census
4700 Silver Hill Road
Washington, DC 20233-9100
(new!) ph: 301-763-1683
FAX: 301-457-2299
e-mail: [log in to unmask] or [log in to unmask]
-------------------------------------------------------------------
This list is for discussion of modelling issues and the BUGS software.
For help with crashes and error messages, first mail [log in to unmask]
To mail the BUGS list, mail to [log in to unmask]
Before mailing, please check the archive at www.jiscmail.ac.uk/lists/bugs.html
Please do not mail attachments to the list.
To leave the BUGS list, send LEAVE BUGS to [log in to unmask]
If this fails, mail [log in to unmask], NOT the whole list
|