Dear Andrew M.,

I think you're right. The point is that, for this example, the seemingly noninformative gamma(1,1) prior distribution does very strong things to the inferences, whereas the seemingly noninformative uniform(0,1000) does not do such strange things.

One way to look at it mathematically is to consider noninformative prior distributions as limits. For example, gamma(a,a), with a approaching 0, or uniform(0,A), with A approaching infinity. In this example (and others), the gamma(a,a) prior distribution does not lead to nice posterior distributions in the limit "a approaches 0". In contrast, the uniform(0,A) prior distribution gives stable posterior distributions in the limit "A approaches infinity". For example, using a prior distribution of uniform(0,100) will give essentially the same results as using uniform(0,1000). In contrast, the gamma prior distribution does weird things and, since it is the default in BUGS analyses, I thought it would be a good idea to alert the newslist.

Yours,
Andrew G.

Andrew Millard wrote:

[log in to unmask]">

I think the problem here is that the gamma(1,1) prior on the precision is
a poor attempt at being uninformative.  Whilst the Unif(0,1000) prior on
the standard deviation expresses prior knowledge that sigma.theta could
equally well take any value up to 1000, the Gamma(1,1) prior effectively
says that our prior knowledge is that sigma.theta is unlikely to be more
than 6, and is more likely to be close to 1.

Reading the version of the "8 schools"  example in the first edition of
the book (sorry I don't have the 2nd edition yet), it is clear that the
prior knowledge of the likely values is not well represented by either of
these distributions: the y are within-school mean increases with coaching
of a score that (on a large scale and for individuals) lies between 200
and 800 with mean 500 and s.d. 100.  It would seem that prior distribution
for the between-school s.d. of the increase, sigma.theta, should be mostly
below 100 and favour smaller values over larger ones.  The question is
then to choose an appropriate distribution to represent that knowledge.
If conjugacy is really necessary for some reason then perhaps a
Gamma(0.5,4) prior on the precison might be suitably vague.

I don't believe there is such a thing as a completely uninformative prior,
though some are vaguer than others.  An important question in all Bayesian
analyses is how to represent our prior knowledge, and if that prior
knowledge is very uncertain, we need to think about just how uncertain it
is, rather than unthinkingly choosing a prior which has been labelled as
uninformative by someone else, whose problem is different and may be on a
different scale.

Andrew

On Tue, 29 Jul 2003, Andrew Gelman wrote:

Hi all.  I just wanted to let youall know that problems can occur with
seemingly-noninformative inverse-gamma prior distributions for variance
components.   In many cases, it's more reliable to simply use uniform
prior distributions on the standard deviation parameters.

<snip>

For example, in a simple 1-way data structure (the "8 schools" example
from "Bayesian Data Analysis"), we use the following model:

model {
  for (j in 1:J){
    y[j] ~ dnorm (theta[j], tau.y[j])
    theta[j] ~ dnorm (mu.theta, tau.theta)
    tau.y[j] <- pow(sigma.y[j], -2)
  }
  mu.theta ~ dnorm (0, 1.0E-6)
  tau.theta <- pow(sigma.theta, -2)
  sigma.theta ~ dunif (0, 1000)
}

This is a uniform prior distribution on the the sd parameter
(sigma.theta) and works fine.

An alternative uses the inverse-gamma prior, replacing the last 2 lines
in the above model by,

  tau.theta ~ dgamma (1, 1)
  sigma.theta <- 1/sqrt(tau.theta)

This does not work well at all (see Figure C.3 on page 597).


 =========================================================================
 Dr. Andrew Millard                              [log in to unmask]
 Department of Archaeology, University of Durham,   Tel: +44 191 334 1147
 South Road, Durham. DH1 3LE. United Kingdom.       Fax: +44 191 334 1101
                     http://www.dur.ac.uk/a.r.millard/
 =========================================================================

-------------------------------------------------------------------
This list is for discussion of modelling issues and the BUGS software.
For help with crashes and error messages, first mail [log in to unmask]

To mail the BUGS list, mail to [log in to unmask]
Before mailing, please check the archive at www.jiscmail.ac.uk/lists/bugs.html
Please do not mail attachments to the list.

To leave the BUGS list, send LEAVE BUGS to [log in to unmask]
If this fails, mail [log in to unmask], NOT the whole list

------------------------------------------------------------------- This list is for discussion of modelling issues and the BUGS software. For help with crashes and error messages, first mail [log in to unmask]

To mail the BUGS list, mail to [log in to unmask] Before mailing, please check the archive at www.jiscmail.ac.uk/lists/bugs.html Please do not mail attachments to the list.

To leave the BUGS list, send LEAVE BUGS to [log in to unmask] If this fails, mail [log in to unmask], NOT the whole list