Dear SPM-list,
I have a quick question about how to do multiple-comparisons correction
on a dataset consisting of a bunch of behavioural measures
(reading scores, etc.) and fMRI measures (average activities from
various ROIs, in various conditions).
Just to be clear, this is not a question about doing multiple comparisons
across a volume, where all the Gaussian fields stuff would apply.
It's just a case of correlating a bunch of vectors with a bunch
of other vectors.
I want to correlate all the behavioural measures
with all the fMRI measures, and then choose a suitably
high treshold to correct for the fact that I'm making
lots of multiple comparisons.
A Bonferroni correction would divide all the p-vals
by ( num_behav_measures * num_ROIs * num_conditions )
which is a large number.
However, many of the measures are highly correlated
with other measures from the same class (e.g. the behavioural
measures include several separate reading scores, which
are all quite correlated with each other, but which all
get thrown in to the big correlate-everything-with-everything mix).
So, the effective number of correlations that I am running
is somewhat less than the vanilla Boferroni divisor.
But how much less? I don't know.
I expect that this situation arises fairly often,
but I haven't been able to find a formula for calculating it.
If you happen to know one, I'd be very interested to hear it.
Failing that, I wrote a short Matlab program
that tries to use a permutation test to do the
multiple comparisons correction.
I believe that it follows the standard permutation test logic,
but I'd be very grateful to hear the opinion of someone more knowledgeable.
Here's what the code does:
In the null hypothesis, there is no true relationship
for any given subject between his behavioural data and his fMRI data.
In that case, we could shuffle the data across
the different subjects, and it wouldn't make any difference.
If we shuffle enough times, and collect all the resulting
correlations, then we'll get the distribution of correlations
that you would get from correlating all the fMRI scores
with all the behavioural scores, under the null hypothesis
that there is no true relationship between them.
Here's the code. It's extremely fully commented,
so hopefully it's fairly easy to read.
http://faculty.washington.edu/raizada/raj_permutation_test_behav_fMRI.m
If you want to run the program, you'll need the data,
which is in this file:
http://faculty.washington.edu/raizada/raj_values_for_permutation_test.mat
I'd be very grateful for any feedback about whether
the stats look valid or not.
Many thanks in advance for any comments or suggestions,
Raj
-------------------------------------------
Rajeev Raizada, Ph.D.
Postdoctoral Research Fellow
Institute for Learning and Brain Sciences
University of Washington
Box 357988, Seattle WA 98195
E.mail: [log in to unmask]
Tel: 206 221 6415
Fax: 206 221 6472
|