Alex,

> 
> Interestingly, however, we found that, when looking at uncorrected
> p-values, CBF in the left amygdala correlated with anxiety scores within
> MDMA scans, but also when pooling MDMA and Placebo scans. We thought that
> the fact that this was the only regional correlation for which this was the
> case may be significant. We included this finding in a manuscript and since
> we did not want to report just any correlations that were significant based
> on uncorrected statistics, we applied a systematic criterion regarding
> which correlation results were reported: Correlations reported had to be
> present *both* when 
> 
> - including only MDMA scans into co-variate analysis and
> - pooling MDMA and Placebo scans in the co-variate analysis
> 
> Of course, this criterion is data-driven and thus somewhat post hoc, but we
> still think that it is a reasonable limitation in order to counteract the
> lack of a correction for multiple comparison. We think that it is plausible
> to assume that a true correlation between brain activity and some
> psychological variable (basically a "neural correlate of consciousness")
> should be present over a range of conditions, including placebo and
> pharmacological stimulation. Thus, in our case, we think that it is not
> unreasonable to require that it should be observable within MDMA scans AND
> within the pooled data set including MDMA + Placebo scans.
> 
> Since we ran into some criticism from the reviewers, I'd like to ask you
> for your opinion, particularly on these questions:
> 
> - do you think that the presence of the correlation between amygdalar CBF
> and anxiety both within MDMA scans alone AND within the pooled data set
> (MDMA+ Placebo), even if significant only using uncorrected statisticis, is
> meaningful and has any informative value?

You'd have to define what you mean by "meaningful". The answer from a 
map-wise false positive rate control perspective, however, is straightforward:
what you want to determine is the probability of seeing your two results or 
greater values in the spatial dataset. This would be straightforward to 
determine if the two correlations were independent and the voxel values
across space were independent. However one thing 
which you might or might not have realized is that your two correlations 
are correlated (because the values from the NMDA condition are shared in 
both). This makes common behavior between the two less surprising than if 
they were independent values. And of course there is the spatial
correlation.

> 
> - if so, do you think the chosen criterion for counteracting the lack of
> correction for multiple testing is reasonable or do you have any other
> suggestions?

Like I mentioned above, from a false positive rate perspective you'd
have to determine the probability of seeing your pair of results in the 
dataset. You could, of course, determine an upper bound on this probability
and see if this is lower than the desired false positive rate.

I sensed from other parts of your message that you had some prior beliefs
about what you should see. If you want to explicitly incorporate
these prior beliefs into your analysis, you should use a Bayesian
approach. Bayesian inference is, if I may indulge in a bit of melodrama,
in a different conceptual universe from false positive rate control. So if
you decide to use your prior beliefs about the existence of correlations
in your analyses, I'd advise that you make your inferential framework 
clear from the outset of your ms.


Sincerely,
Eric


Eric Zarahn
Columbia University

> 
> 
> Thanks very much
> 
> Alex Gamma
> 
> 
> University Hospital of Psychiatry
> Research Unit
> Lenggstr. 31
> 8029 Zurich
> Switzerland
> 
> Email: [log in to unmask]
> Phone: +41 1 384 26 32
> Fax: +41 1 384 33 96
> 



%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%