 |
 |
Thank you very much for your answer. > Haven't done anything so far, mostly because of this paper [1], Troendle > et al. While this paper worked in microarrays and not smooth images, > they did do simulations in the small n, large number of tests setting, and > found that the Bootstrap was sometimes invalid, sometimes conservative, > depending on the resampling method; if you stick with the valid method, > you will have better power by going with thea permutation method (which > is exact). (I'd be happy to send a copy of the paper). This is interesting. It's very kind of you to offer to send a copy. We no longer have a subscription to that Journal, so actually I'd appreciate the copy (if it is not too much bother). > > > We tried several bootstrapping schemes as part of our ADC map > > investigation, but I am not sure that if we wrote them up we would be > > reporting anything new. > > If you found that the Bootstrap was useful for finding Familywise Error > (max-based) thresholds, then yes, it would be useful to report. It would > be useful because: (a) since the Bootstrap is only approximate, it needs > to be shown that it works in a given context, and (b) if only because > it contradicts existing work [1]. No, we did not find anything that worked. We found that the bootstrap performs poorly in all schemes we tried. Actually, the task was not testing between groups but, more basically, estimating the proportion of fields with maxima over the given threshold (with a view of estimating the 'empirical null'). We looked at the bootstrap estimate of the empirical distribution of the maxima (the histogram of the maxima in the bootstrap samples), and it just did not approximate the the distribution obtained by large-sample simulations well. In their book on the Bootstrap, Efron and Tibshirani mention that accurate estimation of the tails on an empirical distribution of the statistic of interest is a task in which the bootstrap is known to perform poorly [1, p.81]. They make the example of estimating the upper bound of a uniform distribution on a finite interval. I concluded that extimating the distribution of maxima of random fields might be another case belonging to this category. I was puzzled, however, because the distribution of the maxima could be reasonably approximated when we swapped 'sampling with replacement' for 'sampling without replacement'. Because of the role of sampling with replacement in using the sample at hand as a maximum likelihood estimate of the empirical distribution, this should not be so and I do not understand it. [1] Efron B, Tibshirani RJ 1993. An Introduction to the Bootstrap. London: Chapman & Hall. > > I look forward to seeing you results. As said, I did not find anything that worked. But you are certainly wellcome to have a look at the submitted tech report where some of this stuff has been included if you are interested. Unfortunately it contains a lot of other material -- the bootstrap stuff is in the appendix. Roberto