Hi all,
Here is the summary of all the responses I received regarding the
possibility of testing for circular equivalence. I wanted to thank
everyone for the clear but detailed responses (particularly from Bill
Jefferys). I definitely received enough information to get my bearings
(haha) in this problem. The consensus is: 1) Bayesian analysis can
definitely be used for testing for circular equivalence, provided one can
come up with a sensible definition of what 'equivalence' is, and 2) there
are special considerations with regard to comparing two measurement methods
which need to be taken into account in the analysis.
Thanks again everyone for your input. My first summary, which includes the
first round responses, and second round responses are copied in
below. [the most recent responses are therefore @ the very bottom of the msg]
Regards,
Russell
________________________________
FIRST SUMMARY
________________________________
Hi all,
A week ago, I asked a question regarding the possibility of using Bayesian
stats to test for equivalence of circular data. Thank you all for your
responses (copied in below) -- there was a wide variety of suggestions, and
I've alot to chew on now! Unfortunately, there was some disagreement about
whether a useful Bayesian test can be applied here. I think that I should
try rephrasing the question.
The data are circular: angles around a circle, from zero to 2*pi. Example:
a sample of 40 current flow compass headings. Generally, I have 2 samples
of headings which I want to compare and test whether the mean headings for
their underlying populations are the same (a test of equivalence, with H0:
means are different; Ha: means are the same).
Example 1: paired samples of 40 current headings, one measured by a
hideously laborious manual method versus the other measured by an
easy-breezy automatic method. I'm testing for equivalence of the 2 methods.
Example 2: a paired sample of 40 current headings, one measured at one
place underwater versus another measured about 3 meters away. Test: are the
current headings statistically the same at the 2 locations? Again, the
goal is to test for statistical equivalence, NOT test for statistical
difference, because I want to be able to use the conclusion that the
current flow is the _same_ across 3 meters of space in my subsequent work.
Some responders suggested that Bayesian stats would work here, because they
compute both the probabilities of equivalence & difference. Others said
the opposite, or that Bayesian stats wouldn't necessarily work so easily,
and that alternative methods might be more appropriate.
Anyone feel like wading in? My personal impression is that 1) Bayesian
stats ought to work here, 2) no one's done this before [dammit] 3)I need to
read up on how to use Bayesian stats!
I've summarized the references I received below -- if anyone has anything
to add, please let me know, and the original responses are also at the
bottom of the msg.
Thanks for all the input,
Russell
________________________________________________________________________________________________
Suggested References:
For Circular Data Analysis
[I'm not familiar with Fisher, but the other 3 do not do equivalence tests]
N. I. Fisher, Ed., Statistical analysis of circular data, Cambridge
University Press, 1993
KV Mardia and PE Jupp Directional Statistics (now in 2nd edition)
Zar 1998, Biostatistical Analysis. [suggested by me]
Oriana Software (metasearch pulled a couple websites selling it)
For Alternative Equivalence Testing Possibilities
Bland JM, Altman DG. Statistical methods for assessing agreement between
two methods of clinical measurement. Lancet 1986;I:307-310.
Kelsall, J and Diggle P. Statistics in Medicine,1995 14:2335-2342
________________________________________________________________________________________________
ORIGINAL RESPONSES:
But seriously, this is Bayesian statistics. Make your own test.
Perhaps the simplest place to start is to reformulate the PINES example to
fit your problem. Let me know if you run into problems.
Finn Krogstad
I'm no expert, but I have done some reading about circular
distributions. I think I understand what was meant by the suggestion. In
Fisherian analysis, you just assume the null hypothesis for convenience,
when you really want to reject it in favor the alternative. But, in
Bayesian analysis, you compute the probability of both the null hypothesis
and the alternative. Therefore, the Bayesian approach would be similar to
what you have already been reading about.
Except that you can reject the alternative in favor of the null, which you
couldn't do before.
Rodney Sparapani
I am a relatively new BUGS user so cannot help with implementation but
wonder if you have looked at the Directional Statistics book by KV Mardia
and PE Jupp. It has two sample test for mean and concentration parameter.
Jack
John H Schuenemeyer
This is not a Bayesian solution but might give you something to try.
Kelsall, J and Diggle P published two papers in 1995 (one in Statistics in
Medicine the other in Bernoulli) that compared two density functions. You
might be able to do this on a circle rather than a line. We used the
approach to compare densities of sea turtle nesting sites
http://www.pnl.gov/statenvi/ssenews/ENVR_5_1.pdf and you could use the same
"random labeling" Monte Carlo tests to find which directions differ between
data sets. Again (as you said), this is a test of difference but the
Kelsall and Diggle approach gives pointwise confidence bands, so if the
ratio of directional densities is always within the random labeling bands,
you conclude no significant difference between the two distributions.
Lance Waller
Not sure what you mean by "circular data". And I'm just beginning BUGS. But
I won't let this stop me answering your question on sea slugs! It strikes
me that an equivalence test is entirely straightforward from the Bayesian
route (and surprisingly torturous otherwise).
You have a bunch of parameters which includes d = the difference (ratio,
whatever) between one method (expensive measurement) and the other (cheap
measurement). You want to test whether d is "about zero". Define what this
means (i.e. define your equivalence interval, [a,b] say. Now calculate the
posterior for d, and see what proportion of the area is over H0: d in [a,b]
versus H1: d not in [a,b].
Patrick Johnston.
I was just thinking about it (sea slugs...even cooler than turtles?).
Equivalence testing is a tough problem that I'm not sure becomes easier if
you are a Bayesian. I would suggest a variant of Kelsall and Diggle around
a circle rather than along a line.
[Statistics in Medicine,1995 14:2335-2342.]
Lance Waller (via Elizabeth Hill)
Russell, your problem may be analogous to one we run into in doing
psychological studies. There are times when we want to test whether two
groups are equivalent (perhaps before beginning some intervention), which
is different than the traditional test of differences between
groups. Someone published a procedure for doing this in Psychological
Bulletin, probably ten or more years ago. I don't remember more and only
know about it because I was the associate editor handling the
manuscript. I'm not sure whether this will be helpful or not. I've had
some neuroscience friends who studied slugs, and always thought it would be
fun to show up at the lab in the morning by shouting "you slugs!" Good
luck on your project.--Rick Wagner
Russell,
Equivalence testing is hard, which is why you don't find much useful
literature. Here is I method that I discovered (as far as I know,
anyway). It works for any estimate x whose sampling standard deviation s
can be obtained (just read it off the computer output, mostly). Assuming
that 0 is an appropriate null hypothesis value (as in comparing two means),
set up the hypotheses:
H- the parameter is < -s
H0 the parameter is between -s and +s
H+ the parameter is > +s
Choose H- if x<-s, H+ if x>+s, H0 otherwise. This has three handy
features. (1) the 0.05 level test of H- vs. H+ has power 0.95, (2) the
strength of the assertion being made is estimated by the data (essentially
it is 2s, the width of the H0 interval), (3) if your data don't have much
to say, you end up saying nothing (H0). I have a paper under review that
explains this more generally.
Mikel Aickin, Ph.D.
Hi Russel:
I am not sure if I understand your question. You mention that you want to
test for equivalence - does that mean that you want to treat equivalence as
the alternative hypothesis.
If that is the case, it is not a problem in the Bayesian setting as many
Bayesian hypothesis testing procedures treat the null and the alternative
hypotheses interchangeably.
I did some work on Bayesian analysis of circular data and have interest in
this area.
Sanjib Basu
Dear Russell,
I'm not really an expert in this field, but there are books by Mardia and
by Fisher on circular statistics.
Many years ago I worked with wind direction statistics and as far as I
remember you can do very nice things with the von Mises distribution: c *
exp(-a * cos(x - mu)), where mu is the mean direction, a an equivalent for
the inverse of the variance and c a normalizing constant.
See N. I. Fisher, Ed., Statistical analysis of circular data, Cambridge
University Press, 1993. and references there.
There exists a package (non-free) called Oriana.
Also see: http://geography.lancs.ac.uk/cemp/resources/software.htm for
applications in a different field.
I hope this will help.
Paul Eilers
Hi Russell,
This is not a direct answer to your question about Bayesian methods, and
you may already be aware of what I'm suggesting, but there has been some
work on measuring agreement between different methods of clinical
measurement in the medical field - probably in others too. I have no idea
whether it might be adaptable to your data.
The source paper is Bland JM, Altman DG. Statistical methods for assessing
agreement between two methods of clinical measurement. Lancet
1986;I:307-310. I've used the method once in a published study (Desai SP,
Sivakumar S, Fryers PT. Evaluation of a disposable prism for applanation
tonometry. Eye 2001;15:279-282.)
It's not so clear-cut as traditional hypothesis tests to identify a
difference, but is fairly intuitive.
Paul Fryers
**********************************
SECONDARY RESPONSES:
**********************************
The basic testing problem as you've laid it out --
H0: means are different
Ha: means are the same --
has no solution as stated, under Bayesian theory, Neyman-Pearson (i.e.,
classical) theory or any other that I'm aware of. H0 can never be accepted
under this framework. First, you need to decide on a practical difference
D that you want to detect. If the means of measurements by the 2 methods
are within D, then they are regarded as equivalent.
Also, the relative error variance of the measurement techniques is
critically important as well. Equivalence of means (or other location
parameter) certainly does not imply either equivalence of measurement
methods or equivalence of populations.
You seem to be comparing measurement methods, but you haven't said whether
these are paired measurements, observing the same event with 2 different
methods. That would make sense for a methods comparison. ASTM has
published guidance for comparing measurement methods and also for testing
the comparability of sets of measurements.
There are some rather subtle problems here, so you should be careful. Take
your time and get a good theoretical grounding before you go much further.
Best wishes,
John Carson
********************************
Lots of interesting opinions!
Though I don't know how to do the calculations, I remain convinced that a
Bayesian approach would be a good way to go, as it would allow you to
compute the probability that the angle difference is less than +-10 degrees
(or any other range that you consider "equivalent enough") You could even
make a curve of probability against different degrees of difference.
Note that Waller's comment about using an ordinary test, from which you
"conclude no significant difference" suggests the old-fashioned (and
illogical) notion that something that isn't statistically significant isn't
happening. Far too many people still believe that.
The simple way to think about testing is that what you are trying to show
must always be the *alternative* hypothesis. Tests cannot provide evidence
for the null hypothesis.
I don't think of Bayesian analysis in terms of "hypothesis tests" at all.
Rather, they allow you to put (belief) probabilities on any proposition that
is of interest to you.
Dave Parkhurst
*********************************
This should be very straightforward since the circle is a compact space so
even if one presumes uniform priors on the parameters you are testing at
least the calculation is well defined. You will have to decide on a
likelihood function, and I can't help you with that other than to mention
that there is a sort of standard circular distribution that looks normal
near the parameter but is smooth throughout. Here it is: von Mises
distribution, p. 200 of volume I of Kendall, Stuart & Ord.
f(x)={2*pi*I_0(k)}^{-1}*exp(k*cos(x-m))
where the parameter m is the location parameter (an angle) and the
parameter k describes how concentrated the data are. I_0 is the modified
Bessel function of the first kind of order 0 (argh).
I don't know if BUGS can do the calculations; this is a problem where the
dimensionalities of the spaces on which the probabilities live differ under
the two models, so I would generally think a reversible jump approach would
work well...but I had difficulty getting anything but the simplest
reversible jump problems to work under BUGS so I eventually found that
coding the problem in R worked. Moreover, if you sample k you are going to
have to evaluate that Bessel function, and I don't know an easy way to do
that. Probably you'll have to look up an approximation. I doubt that the
series expansion given by Kendall et. al. will be adequate, but I could be
wrong. Abramowicz and Stegun probably has the approximation you need.
A good introduction to reversible jump would be the paper by Dellaportas,
Forster and Ntzoufras:
http://www.maths.soton.ac.uk/staff/JJForster/Papers/ModSelect.html
Of course, von Mises may not be the distribution you want for the
likelihood. That, and the priors, are up to you, as always in Bayesian
analysis.
You may also want to cast this as a decision problem. Are there costs under
the two models? This should be factored in.
Anyway, knowing nothing about your problem [WARNING, this is just for
illustration! You need to make the probability assumptions that YOU
believe, not necessarily the ones I am suggesting below]:
Under model 1, there is one mean m and one "spread parameter" k. Assume von
Mises distribution and uniform prior on m (on [0,2*pi). k goes from 0 to
infinity, so a sort of standard prior on k might be 1/k, but don't take
that to the bank. Think about it. Assuming sampling over k as well as over
m, you need a sampler, probably Metropolis-Hastings, that does first one
and then the other. For the step on k you need to evaluate that Bessel
function. For the step on m, you need to be careful as m "wraps" at 2*pi.
Just make sure you do it right. The data would be combined into a large
single set.
Under model 2, there are two means m1 and m2 and one or two "spread
parameters" k (or k1, k2). You have to make assumptions about that. Is
there reason to think that under this model, both sets of data have the
same k (hopefully yes)? If so, use one k, otherwise use two. If you have to
use two, there will be a dimensionality difference of two between the two
models, which means that you'll need more data to get an equivalently
strong discrimination between the models. Let's say one k suffices. Then
you have to sample on m1, m2 and k. The sampling on m1 and m2 will involve
the data from the two sets separately. The sampling on k will involve the
combined data set.
Then you have to have a reversible jump step. This proposes model 1 or
model 2 (depending on which model you are currently on), and accepts or
rejects the jump depending upon the M-H factor as calculated in the paper
by Dellaportas _et. al._ You also need priors on the models (conventionally
people take it equal on both models, but that may not conform to your
actual prior information, so again, this is something you have to think about).
The result is basically how long the sampler spends on model 1 and how long
it spends on model 2. These will be in proportion to the posterior
probability of the two models, respectively. Again, if there is are
different costs depending on which model is correct, then this is the time
to factor in the loss function.
So, given all this I think that the problem you have posed is certainly
doable, and probably not too difficult to do, although you will have to
think carefully about the assumptions you make, for what I have outlined is
only one particular approach and it may not conform to your prior information.
William H. Jefferys
______________________
Russell Wyeth
Friday Harbor Laboratories
620 University Rd
Friday Harbor, WA
98250
USA
[log in to unmask]
Ph: 206 5431484
Fx: 206 5431273
-------------------------------------------------------------------
This list is for discussion of modelling issues and the BUGS software.
For help with crashes and error messages, first mail [log in to unmask]
To mail the BUGS list, mail to [log in to unmask]
Before mailing, please check the archive at www.jiscmail.ac.uk/lists/bugs.html
Please do not mail attachments to the list.
To leave the BUGS list, send LEAVE BUGS to [log in to unmask]
If this fails, mail [log in to unmask], NOT the whole list
