Dear Bugs list members
Please find below a collection of respones to Finn Krogstad's list message about
new user's questions, which may be of particular interest to those either
teaching or learning Bugs. Thanks to all those who contributed - this discussion
is now closed.
Best wishes
Ken Rice
Bugs list manager
-----------Finn Krogstad----------
I would like to post a general response to the recurring message:
'here is my problem. I am a newbie. Can someone show me how to code
this in BUGS?'
I realize that I may be a stick in the mud, but we really need to tell
people that they need to learn BUGS first, and only then apply it to
their problem. Working through the BUGS examples from simple to more
complex not only provides experience, but demonstrates modeling options
that may be superior to the model you originally intended to use.
It should also be repeatedly pointed out that MCMC simulation is
dangerous. Multi-modal distributions, poor burn-in, lack of convergence
are all issues require considerable experience to identify and avoid.
As such, even if you could provide a newbie with code to exactly fit
their problem, they still lack the experience to avoid even common
pitfalls. (A student once commented that statistical consulting is like
giving a loaded handgun to someone who has never seen one and has no
idea what it is, and then telling them to be careful).
I think that it is useful to inform interested newbies that their
problem can (or can't) be addressed in BUGS, but it is probably not a
good idea to give them code help until they are already fairly
proficient in coding and running models.
I would like to post a general response to the recurring message:
'here is my problem. I am a newbie. Can someone show me how to code
this in BUGS?'
I realize that I may be a stick in the mud, but we really need to tell
people that they need to learn BUGS first, and only then apply it to
their problem. Working through the BUGS examples from simple to more
complex not only provides experience, but demonstrates modeling options
that may be superior to the model you originally intended to use.
It should also be repeatedly pointed out that MCMC simulation is
dangerous. Multi-modal distributions, poor burn-in, lack of convergence
are all issues require considerable experience to identify and avoid.
As such, even if you could provide a newbie with code to exactly fit
their problem, they still lack the experience to avoid even common
pitfalls. (A student once commented that statistical consulting is like
giving a loaded handgun to someone who has never seen one and has no
idea what it is, and then telling them to be careful).
I think that it is useful to inform interested newbies that their
problem can (or can't) be addressed in BUGS, but it is probably not a
good idea to give them code help until they are already fairly
proficient in coding and running models.
Finn Krogstad
University of Washington
College of Forest Resources
Seattle WA 98195
I would like to post a general response to the recurring message:
'here is my problem. I am a newbie. Can someone show me how to code
this in BUGS?'
I realize that I may be a stick in the mud, but we really need to tell
people that they need to learn BUGS first, and only then apply it to
their problem. Working through the BUGS examples from simple to more
complex not only provides experience, but demonstrates modeling options
that may be superior to the model you originally intended to use.
It should also be repeatedly pointed out that MCMC simulation is
dangerous. Multi-modal distributions, poor burn-in, lack of convergence
are all issues require considerable experience to identify and avoid.
As such, even if you could provide a newbie with code to exactly fit
their problem, they still lack the experience to avoid even common
pitfalls. (A student once commented that statistical consulting is like
giving a loaded handgun to someone who has never seen one and has no
idea what it is, and then telling them to be careful).
I think that it is useful to inform interested newbies that their
problem can (or can't) be addressed in BUGS, but it is probably not a
good idea to give them code help until they are already fairly
proficient in coding and running models.
----------Ken Rice-----------------
Dear Finn
Thanks for your message - many list messages of exactly this type are
answered directly by me, the list moderator. While I (invariably!) make
the occassional slip, `newbie' messages that make it onto the list are
generally
1) Topics I know insufficient about to give an authoritative answer
2) People who demand (!) their message be broadcast after being told by me
to learn Bugs properly before they attempt something difficult.
I guess the message which prompted you to write was of type 2 - please
feel free to mail such list members directly with your advice!
Best wishes
Ken Rice
----------Mark Borsuk <[log in to unmask]>-----------------
I have to say, Finn, that I disagree strongly. Statistical modeling in
general (and Bayesian modeling with BUGS in specific) is a practice that
is useful for most, if not all, scientific endeavors, and therefore should
be widely encouraged. I think it is presumptuous to think that someone
should know every detail and option of BUGS before beginning to apply it
(how many of us can ever claim such expertise?). Such a request is
unreasonable for most applied (or basic) scientists who are more
interested in solving their problem than learning every aspect of specific
software. Anyway, often the best way to learn to use a tool is to have a
specific application in mind (How many of us learned to use our word
processing program, for example, by reading the user's manual cover to
cover and then writing two dozen practice documents, before ever writing
our first real manuscript?)
As people "in the know" with Bayesian modeling and BUGS, let's do whatever
we can to encourage general use, and not claim that, as statisticians, we
have a monopoly on the necessary knowledge.
--------- Friederike Barthel <[log in to unmask]>----------
I have to say that I don't agree with you either. I have used a lot of
statistical programs and I would say that it is fairly impossible to be able
to know all the code, functions etc of every program. If I know that I can
use a certain program for an experiment and I have not used it before then I
think it is permissable to ask whether somebody else has got more experience
in that particular field. That does not mean that I do not have any
interested in the rest of BUGS, it just means that I do not have the time to
learn the rest until the deadline of the analysis I am doing.
Yours
Friederike
----------------- "Paul, David A" <[log in to unmask]> ------------------
Perhaps it would be best to say that people using a Listserv
should observe the usual courtesies by searching the archives
(when they exist) and doing their best to solve a problem on
their own before they post; then, having one or more helpful
replies, posting a summary for the benefit of all.
The part about "doing their best" of course is going to vary
from person to person depending on their experience level.
I am definitely a "newbie" at Linux and Xwindows, but the
folks at my local Linux User's Group have been very tolerant
of my obviously unsophisticated questions. If I were, however,
to post a question to the BUGS listserv asking what a prior
was, I should have my Ph.D revoked! The trouble is that
it is difficult to evaluate the level of experience of the
people who post to an international listserv.
I can appreciate Dr. Krogstad's arguments. However, I also
concur with Dr. Borsuk. If the statistics community wants to
become as exclusive as the pure math community (ie, almost
irrelevant) then withholding code until the person needing
it has demonstrated sufficient suffering probably makes
sense. If, on the other hand, we want to disseminate
statistical methods as widely as possible.....
----------------- Andrew Millard <[log in to unmask]> ------------------
I have to say that I agree with bits of both Finn and Mark's comments.
MCMC is dangerous: to use it successfully requires some knowledge of
convergence and it is easy to think that the output has given the solution
to a problem when it hasn't. To use BUGS requires some knowledge of its
specific syntax, coding tricks, and limitations. Unsurpisingly, newbie
questions (and many other questions) on this list focus on the latter
knowledge. We have some discussion of appropriate modelling, how to do
model choice etc, but very little on judging whether one's BUGS output is
reliably estimating posteriors (except for those who try to emulate the
Stagnant example!).
However, Bayesian modelling is powerful and useful in many disciplines.
It also requires some background knowledge of statistics. Finn's comments
suggest to me that he thnks many people are coming new to BUGS *and*
Bayesian nodelling at the same time. I don't know if this is true, but if
this is how people come to BUGS then they have a very steep learning curve
(especially if, like me, they have little formal statistical training).
Many newbie BUGS questions can be answered by the examples or close
reading of the manual. However it can be difficult to find the bit that
answers a particular question if you are not familiar with the software.
I agree very much with Mark when he says:
> As people "in the know" with Bayesian modeling and BUGS, let's do
> whatever we can to encourage general use, and not claim that, as
> statisticians, we have a monopoly on the necessary knowledge.
But we must not encourage general use without making sure that people know
the pitfalls of the MCMC technique. There are large red warnings in the
manual, but it may be that someone who is keen to solve a problem and
thinks they have the tool for it takes little heed of them.
Andrew
---------------- Jim Hodges <[log in to unmask]> ------------------
Mark:
Forgive me for coming down hard on you, but you're just plain wrong. As
the BUGS manual says, MCMC is still experimental, though you could easily
miss that in the general euphoria over its power. I would argue that for
all but fairly simple problems, Bayes is still experimental. Can you
assert with complete confidence that your favorite models and priors
cannot produce a bimodal posterior? (Assuming your favorite models aren't
multiple regressions, that is.)
I've worked with subject-matter researchers in a great variety of fields
for 20 years and God bless their souls, few of them are even up to the
rigors of balanced repeated-measures ANOVA. To suggest that they should
be handed some BUGS code and wished good luck is a prescription for errors
even we don't understand.
Jim Hodges
---------------- Pamela Wolfe <[log in to unmask]> ------------------
I can't resist commenting on this -
I'm an applied statistician and have done a lot of consulting
with physicians and scientists in various disciplines. There are folks who
are much better at manipulating software than understanding the results; we
can't prevent this without imposing a statistical police state. The
conclusion I've reached is that part of my work as a statistician is to
educate my clients to whatever extent they're willing - the key being to
recognize when they need to call their statistician.
On the specific issue of BUGS being dangerous - it is certainly any user's
responsibility to understand the software he uses. There are plenty of
caveats in the documentation.
---------------- Stuart Drucker <[log in to unmask]> ------------------
I find myself mostly agreeing with Finn's argument. While Bayesian modeling
can add significant insight to a discipline, the misuse and lack of
understanding of the mechanics of MCMC can create a backlash against the
technology as well.
In my area, marketing research, MCMC has become a breakthrough technology in
solving choice-based conjoint problems because it directly addressed a
weakness of conventional multinomial logit: assuming homogeneity preferences
because there was rarely enough information at the respondent level to get
individual-level preference functions. We can now achieve dramatically
better predictions, do "on the fly" segmentation tailed to the research
objective and link individual customers to a wealth of database marketing
information.
As exposure to MCMC software grows, there's the danger that marketing
science practitioners (as opposed to formally trained Bayesian
statisticians) will think of it as "fool-proof" and pay superficial
attention to the achievement of convergence, the importance of well-defined
priors, and especially the multi-modal (mixture) distributional problem.
Unfortunately, some of the commercially available software in our
discipline, while a seminal development in extending MCMC to applied
marketers, offers few tools for addressing these issues. That's not to say
that it's unusable, but has to be used by experienced researchers savvy with
the steps needed to build a robust model. The risk that we run is that
under-educated researchers will think MCMC can solve _any_ problem because
it's can be run without crashing, not because the data supports it.
I also worry that some researchers will "leap" to use MCMC to solve problems
that should _not_ be solved by a Bayesian method except for special cases.
For example, the idea of building a customer loyalty or "derived importance"
model for an individual consumer through regression is appealing. However,
there are rarely cases in custom survey research where we have more
observations/brands rated per respondent than the number of parameters.
While there's been some published work to suggest that "reasonable" results
can be achieved if there's as few as three observations per respondent and
five parameteres to be estimated _and_ one has "well-defined" priors, I just
ran across a newsletter where it was suggested that one could model a 1:2
ratio of observations to parameters, and get regression weights for any set
of individual indicators "even those seemingly measuring the same thing by
removing....multicollinearity"!
The software that is likely to be have been used for this problem (not BUGS)
assumes flat priors as a default, an option that is difficult to change.
That flies against the intuition that one should have at least as many
observations as parameters to even make a Hierarchical Bayesian model
"identified".
That's exactly the kind of misuse we have to be on-guard for. The bottom
line is that being able to run a problem is not nearly as important as
understanding when the problem should _not_ be solved as well as when it can
be solved.
---------------- "Anderson, Kevin C" <[log in to unmask]>
------------------
"Statistics is too important to be left to the statisticians."
David S. Moore
---------------- [log in to unmask] ------------------
I would prefer your admonition to read "MCMC can be dangerous";
but in any event, it strikes me as difficult/costly to enforce rules as
to what can or can't appear on the list.
Newbie questions are a mixed blessing -- it means the
methodology is reaching an ever-expanding circle of people, but
as Finn vividly sketches, it also means people are walking around
trying to "be careful" with loaded shotguns, and someone (!) might
have to intervene.
This suggests to me:
(1) the need for a text on Bayes/MCMC that works for the audience
that the methodology is now starting to reach; e.g., grad students
and applied folks in fields like polisci (my own), sociology,
demography, education, who aren't methodological specialists.
Jeff Gill (a political scientist) has a book coming out with C&H in
April 2002 aimed at this type of reader.
(2) the importance of Finn's point that people really ought to
exhaust the resources in the software itself before turning to the
list. The supplied examples cover a tremendous amount of
modeling situations, nifty reparameterizations and BUGS coding
tricks. I wonder if a concept index (e.g., "hierarchical centering") or
even a "trick" index (e.g., "double subscripting") might be a helpful
addition to the examples/help (hi there DS, AT, NB)?
-- Simon Jackman
-------------------------------------------------------------------
This list is for discussion of modelling issues and the BUGS software.
For help with crashes and error messages, first mail [log in to unmask]
To mail the BUGS list, mail to [log in to unmask]
Before mailing, please check the archive at www.jiscmail.ac.uk/lists/bugs.html
Please do not mail attachments to the list.
To leave the BUGS list, send LEAVE BUGS to [log in to unmask]
If this fails, mail [log in to unmask], NOT the whole list
|
