Hi Raj,
thanks a lot for sharing your ideas about the permutation tests. Maybe you are
also interested in an SAS script that has been written by Warren Bilker for
testing correlated correlations, which is also based on permutation tests.
This can be found at:
http://www.cceb.upenn.edu/main/people/bilker.html
All the best,
Thilo
On Wednesday 19 April 2006 22:55, Rajeev Raizada wrote:
> Dear SPM-list,
>
> I have a quick question about how to do multiple-comparisons correction
> on a dataset consisting of a bunch of behavioural measures
> (reading scores, etc.) and fMRI measures (average activities from
> various ROIs, in various conditions).
> Just to be clear, this is not a question about doing multiple comparisons
> across a volume, where all the Gaussian fields stuff would apply.
> It's just a case of correlating a bunch of vectors with a bunch
> of other vectors.
>
> I want to correlate all the behavioural measures
> with all the fMRI measures, and then choose a suitably
> high treshold to correct for the fact that I'm making
> lots of multiple comparisons.
>
> A Bonferroni correction would divide all the p-vals
> by ( num_behav_measures * num_ROIs * num_conditions )
> which is a large number.
> However, many of the measures are highly correlated
> with other measures from the same class (e.g. the behavioural
> measures include several separate reading scores, which
> are all quite correlated with each other, but which all
> get thrown in to the big correlate-everything-with-everything mix).
> So, the effective number of correlations that I am running
> is somewhat less than the vanilla Boferroni divisor.
> But how much less? I don't know.
>
> I expect that this situation arises fairly often,
> but I haven't been able to find a formula for calculating it.
> If you happen to know one, I'd be very interested to hear it.
>
> Failing that, I wrote a short Matlab program
> that tries to use a permutation test to do the
> multiple comparisons correction.
>
> I believe that it follows the standard permutation test logic,
> but I'd be very grateful to hear the opinion of someone more knowledgeable.
> Here's what the code does:
> In the null hypothesis, there is no true relationship
> for any given subject between his behavioural data and his fMRI data.
> In that case, we could shuffle the data across
> the different subjects, and it wouldn't make any difference.
> If we shuffle enough times, and collect all the resulting
> correlations, then we'll get the distribution of correlations
> that you would get from correlating all the fMRI scores
> with all the behavioural scores, under the null hypothesis
> that there is no true relationship between them.
>
> Here's the code. It's extremely fully commented,
> so hopefully it's fairly easy to read.
>
> http://faculty.washington.edu/raizada/raj_permutation_test_behav_fMRI.m
>
> If you want to run the program, you'll need the data,
> which is in this file:
>
> http://faculty.washington.edu/raizada/raj_values_for_permutation_test.mat
>
> I'd be very grateful for any feedback about whether
> the stats look valid or not.
> Many thanks in advance for any comments or suggestions,
>
> Raj
> -------------------------------------------
> Rajeev Raizada, Ph.D.
> Postdoctoral Research Fellow
> Institute for Learning and Brain Sciences
> University of Washington
> Box 357988, Seattle WA 98195
> E.mail: [log in to unmask]
> Tel: 206 221 6415
> Fax: 206 221 6472
--
Thilo Kellermann
RWTH Aachen University
Pauwelstr. 30
52074 Aachen
Tel.: +49 (0)241 / 8089977
Fax.: +49 (0)241 / 8082401
E-Mail: [log in to unmask]
|