I would suggest that reducing the answer to the single mode would be less appropriate than acknowledging that there are 2 reasonable answers. One is twice as likely as the other, but there is still a good chance the alternative is true.
If that is not an option, and you have to make a choice, I suggest you then need to consider the costs of making a wrong decision.
Warren Schlechte
-----Original Message-----
From: Yuanlong Shao [mailto:[log in to unmask]]
Sent: Tuesday, August 16, 2011 12:24 AM
To: Jim Hodges
Cc: Warren Schlechte; [log in to unmask]; Brian J Reich
Subject: Re: [BUGS] Is Multi-Modality a common experience?
Hi Prof. Hodges,
Thank you for the very informative reference.
I'm trying to understand it. The explanation
about conflict prior and likelihood is cool.
But now I guess my question is more about
how to interpret the result from Gibbs sampling.
Take the wet grass example from Kevin Murphy:
http://bnt.googlecode.com/svn/trunk/docs/Figures/sprinkler.gif
If observed that W=T, that grass is wet, then
the posterior probability P(R=T|W=T) is close to 0.5,
which is close to the estimation from Gibbs sampling.
it means that when W=T, our answer to
"whether it was raining" is "not sure at all".
But if you look at the joint posterior probability:
C S R P(C,S,R|W=T)
-------------------------------------------------------
F F F 0
F F T 0.045
F T F 0.18
F T T 0.0495
T F F 0
T F T 0.324
T T F 0.009
T T T 0.0396
There are approximately two modes,
(T, F, T) and (F, T, F), both are very
reasonable explanations for grass to be wet,
with one almost twice higher in probability.
So shall we pick one of them and make
decision then. Say we choose (T, F, T),
then the answer to "whether it was raining"
is "very probably".
After all, if the Bayes net is true, then there
is more chance for a person to see it rained
and then the grass is wet. So when they
saw grass was wet, they may very probably
say that it rained.
Louis
On Fri, Aug 12, 2011 at 7:10 PM, Jim Hodges <[log in to unmask]> wrote:
> I think the answer is "nobody knows in any generality". Here are some
> examples of bimodality that certainly do *not* involve the kind of labeling
> problems that arise in mixture models.
>
> The only systematic study of multimodality I know of is
>
> Liu J, Hodges JS (2003). Posterior bimodality in the balanced one-way
> random effects model. J.~Royal Stat.~Soc., Ser.~B, 65:247-255.
>
> The surprise of this paper is that in the simplest possible hierarchical
> model (analyzed using the standard inverse-gamma priors for the two
> variances), bimodality occurs quite readily, although it is much less common
> to have two modes that are big enough so that you'd actually get a
> noticeable fraction of MCMC draws from both of them. Because the restricted
> likelihood (= the marginal posterior for the two variances, if you've put
> flat priors on them) is necessarily unimodal in this model, the bimodality
> must arise from conflict between the prior and likelihood, but as this paper
> shows, the conflict that produces bimodality is extremely complex.
>
> See also Jon Wakefield's discussion of this paper:
>
> Hodges JS (1998). Some algebra and geometry for hierarchical models,
> applied to diagnostics (with discussion). {\it Journal of the Royal
> Statistical Society, Series B}, {\bf 60}:497--536.
>
> Here a simple, harmless-looking two-level model with normal errors and
> random effect had a bimodal posterior. I don't know what features of the
> data, model, and priors produced this.
>
> My former student Brian Reich also got bimodal posteriors fitting the models
> and data described in this paper:
>
> Reich BJ, Hodges JS, Carlin BP (2007). Spatial analysis of periodontal data
> using conditionally autoregressive priors having two types of neighbor
> relations. {\it Journal of the American Statistical Association},{\bf
> 102}:44--55.
>
> However, those fits don't appear in this paper (long story).
>
> FWIW,
>
> JH
>
>
>
>
>
>
> On 8/12/11 9:41 AM, Warren Schlechte wrote:
>
> I know of a Schnute and Hilborn (1993) paper that might be helpful. It is
> titled "Analysis of contradictory data sources in fish stock assessment" and
> is in the Canadian Journal of Fisheries an Aquatic Sciences.
>
> Here is the abstract:
> Schnute, J.T., and R. Hilborn. 1993. Analysis of contradictory data sources
> in fish stock assessment. Can. J. Fish.
> Aquat. Sci. 50: 1916-1923.
> Fisheries stock assessments sometimes prove, in retrospect, to be wrong.
> Errors may be due to poor model
> assumptions or to data that do not reflect the biological process of
> interest. We develop a method that formally
> admits the possibility of such errors. Likelihood functions derived from
> this method indicate greater uncertainty in
> parameter values than conventional likelihoods, whose derivations presume
> that models correctly describe the
> observed data. The problem of uncertainty is particularly acute when more
> than one data source is available and
> different data sets provide contradictory parameter estimates. Traditional
> methods of stock assessment involve
> weighted averages of the contradictory data, and these generally produce
> parameter estimates intermediate to
> those obtained from the data sets individually. We demonstrate that, when
> model or data errors are considered,
> the most likely parameter values are not intermediary to conflicting values;
> instead, they occur at one of the
> apparent extremes. We provide an example using contradictory trends in
> catch-per-unit-effort data for the Canadian
> northern cod stock (1978-88).
>
> Warren Schlechte
> -----Original Message-----
> From: Yuanlong Shao [mailto:[log in to unmask]]
> Sent: Wednesday, August 10, 2011 10:58 PM
> Subject: Is Multi-Modality a common experience?
>
> Dear List Members,
>
> I know that doing MCMC on mixture models
> has the multi-modality issue due to permutation of labels.
> But is this a common issue in models
> that are not exactly mixtures? Such as those
> models with multiple layers of random variables,
> resulting in a non-convex posterior density surface.
>
> If so, then what additional care do we commonly
> need when making estimations from the samples
> in Gibbs sampling? For mixtures we have various
> ways to deal with label switching, but for a general
> model with multi-modality, do we simply estimate
> parameters by averaging the samples? Or is there
> anyway to restrict the joint samples to be within
> a major posterior density area?
>
> Thanks!
>
> Louis
>
>
>
--
Louis Yuanlong Shao
Department of Computer Science and Engineering
Ohio State University
http://www.shaoyuanlong.com
-------------------------------------------------------------------
This list is for discussion of modelling issues and the BUGS software.
For help with crashes and error messages, first mail [log in to unmask]
To mail the BUGS list, mail to [log in to unmask]
Before mailing, please check the archive at www.jiscmail.ac.uk/lists/bugs.html
Please do not mail attachments to the list.
To leave the BUGS list, send LEAVE BUGS to [log in to unmask]
If this fails, mail [log in to unmask], NOT the whole list
|
