A couple of points:

(1) It is unclear what went into the second level analysis, pre or post images, the difference in pre or post, or one of the images.

(2) It is very circular, you have searched the data to extract, rather than pick an area a priori to extract the data.

Best Regards, Donald McLaren
=================
D.G. McLaren, Ph.D.
Postdoctoral Research Fellow, GRECC, Bedford VA
Research Fellow, Department of Neurology, Massachusetts General Hospital and
Harvard Medical School
Office: (773) 406-2464
=====================
This e-mail contains CONFIDENTIAL INFORMATION which may contain PROTECTED
HEALTHCARE INFORMATION and may also be LEGALLY PRIVILEGED and which is
intended only for the use of the individual or entity named above. If the
reader of the e-mail is not the intended recipient or the employee or agent
responsible for delivering it to the intended recipient, you are hereby
notified that you are in possession of confidential and privileged
information. Any unauthorized use, disclosure, copying or the taking of any
action in reliance on the contents of this information is strictly
prohibited and may be unlawful. If you have received this e-mail
unintentionally, please immediately notify the sender via telephone at (773)
406-2464 or email.



On Tue, Nov 29, 2011 at 4:13 PM, Mario Gatica <[log in to unmask]> wrote:
Dear SPMers,

I would like to ask for an opinion concerning the statistical validity of an analysis I did.

I have pre/post experiment where subjects are scanned before and after an intervention, in order to compare changes in activation. During the scanning sessions subjects were just passively stimulated. This main effect was very robust.

In addition to fMRI I also did psychophysical testing (outside the scanner) and I am trying to find correlations between changes in performance and changes in activation.

I followed the straighforward approach of doing a 2nd level design with the changes in performance entered as a covariate (1 value per subject) and then the contrast "0 1" or "0 -1", as the case may be, limiting the analysis to the postcentral gyrus with a mask (about 1000 voxels). With this design, no voxels survived multiple comparisons correction, so I tried progressively more relaxed thresholds until activation appeared (usually with an uncorrected p of 0.05).  I then  made a spherical mask around this coordinate and extracted the mean raw data from the individual conn* images used in the group analysis (around 30 voxels), and used these to do a Pearsons correlation test outside of SPM (with the psychophysical data), which was positive.

My question is, is this a valid thing to do at all or is the analysis biased? On the one hand, the correlation done outside of SPM  was strongly significant and made sense theoretically. On the hand, this does smell like biasing the analysis. Of course such a thing would not be valid when testing the main effect of interest. Does that also hold for correlation with an external variable (such as behavioural performance)?

Many thanks in advance.

Mario Gatica

--
Mario Gatica
Institut für Neuroinformatik
Ruhr-Universität Bochum