Dear Andrew M.,
I think you're right. The point is that, for this example, the
seemingly noninformative gamma(1,1) prior distribution does very strong
things to the inferences, whereas the seemingly noninformative
uniform(0,1000) does not do such strange things.
One way to look at it mathematically is to consider noninformative prior
distributions as limits. For example, gamma(a,a), with a approaching 0,
or uniform(0,A), with A approaching infinity. In this example (and
others), the gamma(a,a) prior distribution does not lead to nice
posterior distributions in the limit "a approaches 0". In contrast, the
uniform(0,A) prior distribution gives stable posterior distributions in
the limit "A approaches infinity". For example, using a prior
distribution of uniform(0,100) will give essentially the same results as
using uniform(0,1000). In contrast, the gamma prior distribution does
weird things and, since it is the default in BUGS analyses, I thought it
would be a good idea to alert the newslist.
Yours,
Andrew G.
Andrew Millard wrote:
>I think the problem here is that the gamma(1,1) prior on the precision is
>a poor attempt at being uninformative. Whilst the Unif(0,1000) prior on
>the standard deviation expresses prior knowledge that sigma.theta could
>equally well take any value up to 1000, the Gamma(1,1) prior effectively
>says that our prior knowledge is that sigma.theta is unlikely to be more
>than 6, and is more likely to be close to 1.
>
>Reading the version of the "8 schools" example in the first edition of
>the book (sorry I don't have the 2nd edition yet), it is clear that the
>prior knowledge of the likely values is not well represented by either of
>these distributions: the y are within-school mean increases with coaching
>of a score that (on a large scale and for individuals) lies between 200
>and 800 with mean 500 and s.d. 100. It would seem that prior distribution
>for the between-school s.d. of the increase, sigma.theta, should be mostly
>below 100 and favour smaller values over larger ones. The question is
>then to choose an appropriate distribution to represent that knowledge.
>If conjugacy is really necessary for some reason then perhaps a
>Gamma(0.5,4) prior on the precison might be suitably vague.
>
>I don't believe there is such a thing as a completely uninformative prior,
>though some are vaguer than others. An important question in all Bayesian
>analyses is how to represent our prior knowledge, and if that prior
>knowledge is very uncertain, we need to think about just how uncertain it
>is, rather than unthinkingly choosing a prior which has been labelled as
>uninformative by someone else, whose problem is different and may be on a
>different scale.
>
>Andrew
>
>On Tue, 29 Jul 2003, Andrew Gelman wrote:
>
>
>
>>Hi all. I just wanted to let youall know that problems can occur with
>>seemingly-noninformative inverse-gamma prior distributions for variance
>>components. In many cases, it's more reliable to simply use uniform
>>prior distributions on the standard deviation parameters.
>>
>>
>>
><snip>
>
>
>>For example, in a simple 1-way data structure (the "8 schools" example
>>from "Bayesian Data Analysis"), we use the following model:
>>
>>model {
>> for (j in 1:J){
>> y[j] ~ dnorm (theta[j], tau.y[j])
>> theta[j] ~ dnorm (mu.theta, tau.theta)
>> tau.y[j] <- pow(sigma.y[j], -2)
>> }
>> mu.theta ~ dnorm (0, 1.0E-6)
>> tau.theta <- pow(sigma.theta, -2)
>> sigma.theta ~ dunif (0, 1000)
>>}
>>
>>This is a uniform prior distribution on the the sd parameter
>>(sigma.theta) and works fine.
>>
>>An alternative uses the inverse-gamma prior, replacing the last 2 lines
>>in the above model by,
>>
>> tau.theta ~ dgamma (1, 1)
>> sigma.theta <- 1/sqrt(tau.theta)
>>
>>This does not work well at all (see Figure C.3 on page 597).
>>
>>
>
>==========================================================================
> Dr. Andrew Millard [log in to unmask]
> Department of Archaeology, University of Durham, Tel: +44 191 334 1147
> South Road, Durham. DH1 3LE. United Kingdom. Fax: +44 191 334 1101
> http://www.dur.ac.uk/a.r.millard/
>==========================================================================
>
>-------------------------------------------------------------------
>This list is for discussion of modelling issues and the BUGS software.
>For help with crashes and error messages, first mail [log in to unmask]
>
>To mail the BUGS list, mail to [log in to unmask]
>Before mailing, please check the archive at www.jiscmail.ac.uk/lists/bugs.html
>Please do not mail attachments to the list.
>
>To leave the BUGS list, send LEAVE BUGS to [log in to unmask]
>If this fails, mail [log in to unmask], NOT the whole list
>
>
-------------------------------------------------------------------
This list is for discussion of modelling issues and the BUGS software.
For help with crashes and error messages, first mail [log in to unmask]
To mail the BUGS list, mail to [log in to unmask]
Before mailing, please check the archive at www.jiscmail.ac.uk/lists/bugs.html
Please do not mail attachments to the list.
To leave the BUGS list, send LEAVE BUGS to [log in to unmask]
If this fails, mail [log in to unmask], NOT the whole list
