 |
 |
Hi Tom, Welcome to the UK! > Forgive me if this question is totally off the mark, but in the case > where there are few permutations, and thus the estimated null > distribution (say of maximum t-stats or cluster sizes) is "blocky", > how valid would it be to fit an extreme value distribution function to > the blocky null distribution, and then read off the p=0.05 value from > the fit function? I realise that the answer to this question is likely > to be "not strictly valid", but is that something that could be useful > in certain situations? The idea has been toyed with. E.g., in genetics, see the work by Dudbridge and Koeleman, Am J Hum Gen, 75:424-435, who fit extreme value distributions to permutation samples. It's not exactly applicable, and an essential difference is that they have lots of samples to fit to. My intuition is that, with only a handful of permutations, the fit of any parametric distribution to the permutation values will be very poor and unstable. More generally, might it improve the reliability of permutation > testing in situations that give rise to "blocky" null distributions, > as is often the case with maximum cluster sizes for example? e.g. if > one were to iterate through sub-samples of a true null distribution, > and with each sub-sample perform permutation tests to construct an > estimated null distribution, would the p=0.05 values taken from the > permuted distribution be more representative of the true p=0.05 value > than those obtained by fitting an extreme value function first, and > reading off the p=0.05 value from that? > My intuition is that you always lose when you fractionate data, and so I don't think this would help. The direction that might help in some instances is to pool over space. E.g., uncorrected cluster size does this, but with the (not always tenable) assumption of stationarity. One could also pool uncorrected voxel-wise permutations over space, in which case you only need a tiny number of permutations (e.g. like 10, as BAMM does), but you again assume homogeneous permutation distributions over space. I haven't evaluated this, but often wondered if such space-pooling (for voxel-wise inference at least) is a reasonable thing to do. If you're interested in exploring it I can thresh it out more, and suggest how you'd evaluate the accuracy of pooling vs. not. -Tom ____________________________________________ Thomas Nichols, PhD Director, Modelling & Genetics GlaxoSmithKline Clinical Imaging Centre Senior Research Fellow Oxford University FMRIB Centre