Thanks Anderson,
Inclusion of the column of ones (as a nuisance covariate) to model the intercept made little difference to the uncorrected 1-P, but made a huge difference to the corrected 1-P, and cheered me up considerably. The multiple tests of the 'correction for multiple testing' are the multiple TFCE voxels, and the correction is via the usual Gaussian RFT approach to FWE , right?
To the pointy end: presentation of results. Can I simply isolate clusters in the corrected 1-P using a threshold of 0.95 (or more)
and report the maximum in each cluster as the corrected P for that cluster?
To display cluster extent via an overlay, is it OK to show the uncorrected 1-P where it exceeds 0.95?
It seems to me to be desirable to see peaks of significance within the cluster overlay, but 1-P is not very good for that (0.99 and 0.9999 are hardly distinguishable). Instead, -logP would show this clearly. Is there a good reason NOT to do this?
Leighton
From: FSL - FMRIB's Software Library [mailto:[log in to unmask]] On Behalf Of Anderson M. Winkler
Sent: Monday, 6 May 2013 11:04 PM
PS: I overlooked a sentence. Where you asked:
"out_vox_p_tstat1 (when -x in command) is the (1-P) result from the GLM parametric analysis?"
Yes, this is the uncorrected 1-P for the t statistic, but this does not come from a parametric analysis. Instead, it comes from permutation methods.
2013/5/6 Anderson M. Winkler <[log in to unmask]>
Hi Leighton,
Please, see below
2013/5/6 Barnden, Leighton (Health) <[log in to unmask]>
Thanks Anderson,
Am I correct then in concluding that for 'randomise -T' in my simple regression [-1 0 0] case:
out_tfce_p_tstat1 is the permutation (1-P) result?
Yes, that's right, this the 1-P result for the TFCE statistic, not corrected for multiple testing. TFCE is a statistic that combines height and spatial extent of signals.
out_tfce_corrp_tstat1 file doesn't hold useful information?
No no, this holds very useful information. This contains the 1-P for the TFCE statistic after correcting for multiple testing.
out_vox_p_tstat1 (when -x in command) is the (1-P) result from the GLM parametric analysis?
Yes, this is the uncorrected 1-P for the t statistic.
What does out_tstat1 hold?
This holds the t statistic, not p-values.
To extract multi-cluster information, I should therefore replace grot_tfce_corrp_tstat1 with grot_tfce_p_tstat1
in the example in the documentation:
"fslmaths grot_tfce_corrp_tstat1 -thr 0.95 -bin -mul grot_tstat1 grot_thresh_tstat1
cluster --in=grot_thresh_tstat1 --thresh=0.0001 --oindex=grot_cluster_index
--olmax=grot_lmax.txt --osize=grot_cluster_size"
By doing so you are replacing the corrected for the uncorrected p-values. I think most people are interested in corrected results, so I wouldn't make any change, unless perhaps to explore a bit the results.
In one run there showed
"Warning: All design columns have zero mean - consider using the -D option to demean your data"
Ignoring it seemed to work!
You should include the -D, or include a column of ones to account for the intercept. Failing to do so won't give any errors (so, it works), but be aware that the regression line will cross the ordinate (y-axis) at zero, which may be a less-optimal model.
All the best,
Anderson
Thanks for your patience as I inch forward on this, Leighton
From: FSL - FMRIB's Software Library [mailto:[log in to unmask]] On Behalf Of Anderson M. Winkler
Sent: Wednesday, 1 May 2013 7:59 PM
To: [log in to unmask]
Subject: Re: [FSL] randomise defeated by nuisance covariate(s)?
Hi Leighton,
See below:
2013/5/1 Barnden, Leighton (Health) <[log in to unmask]>
Thanks Anderson,
Didn’t read carefully – as soon as I saw residual I thought ‘noise’ as is usually the case for a full (parametric) GLM analysis. Maybe adding thus might help others
"The images are first fit to the nuisance effects alone and nuisance-only residuals are formed (images with the nuisance effects removed). These images are permuted [...]"
I have run ‘randomise –T’ OK and the 1.0-P results (out_tfce_p_tstat1.nii) look as expected, but the T image (out_tstat1.nii.gz) doesn’t seem to make sense, eg where 1.0-P = 0.998 the same voxel has T = 0.0229. What am I missing here?
You are looking at the p-values computed for the TFCE statistic (produced with the option "-T"), and trying to see if it looks similar to the p-value for the t-statistic in the same voxel. The is no reason to expect them to match because they are different things.
If you'd like to compute voxelwise p-values for the t-statistic, use the option "-x". Even so, the p-value won't be identical to what you would find looking at the standard t-distribution. The reason is that randomise tests are based on permutation, and do not use the t-distribution to compute p-values, even when using t-tests or the t-statistic.
All the best,
Anderson
Thanks Leighton
____________________________________
Hi Leighton,
This excerpt discusses permuting the data, or equivalently, the rows of the design matrix. When there are nuisance variables, this procedure is questionable (in fact, it remains valid for most practical cases, but it's still questionable on theoretical grounds). However, if you continue reading the page you see that randomise deals with covariates using a different method. In the same page you'll read:
"The data are first fit to the nuisance effects alone and nuisance-only residuals are formed. These residuals are permuted [...]"So it's not the exchangeability of the data or design that matters, but the exchangeability of the residuals once the covariates have been taken into account. So, you are fine with the design that you described as long as the subjects are unrelated to each other and you don't have repeated measurements, as these would require different treatment.
Note however, that you can't compute "TFCE for clusters". Or you compute stats for clusters, or you use TFCE. Can't mix them.
Hope this helps.
All the best,
Anderson
2013/4/26 Michael Harms <[log in to unmask]>
HI Leighton,
If you want to understand how randomise works in the presence of nuisance
covariates, I suggest you read the statistics papers cited on the
randomise web page, as it is rather technical, and not easy to explain.
cheers,
-MH
--
Michael Harms, Ph.D.
-----------------------------------------------------------
Conte Center for the Neuroscience of Mental Disorders
Washington University School of Medicine
Department of Psychiatry, Box 8134
660 South Euclid Ave. Tel: 314-747-6173
St. Louis, MO 63110 Email: [log in to unmask]
On 4/25/13 6:45 PM, "Leighton Barnden" <[log in to unmask]>
wrote:
>With your help I now have the correct data & covariate input formats for
>randomise. The next hurdle could be trickier though. I want to generate
>TFCE stats for clusters from a simple regression (with severity),
>adjusting for 2 nuisance covariates (global signal level and age). From
>the 'randomise' (theory) wiki :
>
>"Permutation tests rely on an assumption of exchangeability; with the
>models above, the null hypothesis implies complete exchangeability of the
>observations. When there are nuisance effects, however, the null
>hypothesis no longer assures the exchangeability of the data (e.g. even
>when the null hypothesis of no FA difference is true, age effects imply
>that you can't permute the data without altering the structure of the
>data)."
>
>This seems to imply that when the scan order is permuted by randomise the
>order of the nuisance covariates does not follow. If the nuisance
>covariates were always attached to the same scans would not this satisfy
>'complete exchangeability of the observations'? Could this be made an
>option (the default?) for handling of nuisance covariates by randomise?
>Incidentally, inclusion of '--tfce_H 2 --tfce_E 0.5 --tfce_C 6' on the
>randomise command line causes it to fail (FSL5.0).
>Thanks Leighton