Hi Leighton,


2013/5/8 Barnden, Leighton (Health) <[log in to unmask]>

Thanks Anderson,

 

Inclusion of the column of ones (as a nuisance covariate) to model the intercept made little difference to the uncorrected 1-P, but made a huge difference to the corrected 1-P, and cheered me up considerably. The multiple tests of the 'correction for multiple testing' are the multiple TFCE voxels, and the correction is via the usual Gaussian RFT approach to FWE , right?


No no. There is no Gaussian RFT in randomise. The correction for multiple testing is just what it is -- correction for multiple testing. TFCE are statistics that are computed for each voxel, blending intensity with spatial extent of the signal in relation to the noise. See the documentation and the paper on TFCE to for how it works.
 

To the pointy end: presentation of results. Can I simply isolate clusters in the corrected 1-P using a threshold of 0.95 (or more)


Yes, you can do this easily with fslmaths or with the cluster command.

 

and report the maximum in each cluster as the corrected P for that cluster?


No. The maximum 1-P for a cluster (or the minimum P-value inside it) is not the same as the P-value for the cluster itself. If you'd like the P-values for the clusters, use the relevant options in randomise for cluster extent and/or cluster mass. This will require a threshold to be given in terms of the statistic, which isn't the same as the 0.95 as you'd like from the corrected P-values. There is a workaround for this (i.e. to use a non-parametric p-value as the cluster-forming threshold, then compute non-parametric cluster-level statistics), but I don't think it worth the all effort and disk space needed. You can simply use as cluster-forming threshold something as t=2.3 in randomise, and everyone will understand exactly what it means. And the result will of course be valid.

 

To display cluster extent via an overlay, is it OK to show the uncorrected 1-P where it exceeds 0.95?


Uncorrected P-values can be misleading. It's not "wrong" to show them as long they are properly described, but most research questions should be answered with corrected P-values.

 

It seems to me to be desirable to see peaks of significance within the cluster overlay, but 1-P is not very good for that (0.99 and 0.9999 are hardly distinguishable). Instead, -logP would show this clearly. Is there a good reason NOT to do this?


It's fine and probably recommendable to use -log(P). If you use fslmaths, remember to change the basis, i.e., divide the result by ln(10).

All the best,

Anderson



 

Leighton

 

From: FSL - FMRIB's Software Library [mailto:[log in to unmask]] On Behalf Of Anderson M. Winkler
Sent: Monday, 6 May 2013 11:04 PM


To: [log in to unmask]
Subject: Re: [FSL] randomise defeated by nuisance covariate(s)?

 

PS: I overlooked a sentence. Where you asked:

"out_vox_p_tstat1 (when -x in command) is the (1-P) result from the GLM parametric analysis?"

Yes, this is the uncorrected 1-P for the t statistic, but this does not come from a parametric analysis. Instead, it comes from permutation methods.

 

2013/5/6 Anderson M. Winkler <[log in to unmask]>

Hi Leighton,

 

Please, see below

 

2013/5/6 Barnden, Leighton (Health) <[log in to unmask]>

Thanks Anderson,

 

Am I correct then in concluding that for 'randomise -T' in my simple regression [-1 0 0] case:

 

out_tfce_p_tstat1 is the permutation (1-P) result?

 

Yes, that's right, this the 1-P result for the TFCE statistic, not corrected for multiple testing. TFCE is a statistic that combines height and spatial extent of signals.

 

out_tfce_corrp_tstat1 file doesn't hold useful information?

 

No no, this holds very useful information. This contains the 1-P for the TFCE statistic after correcting for multiple testing.

out_vox_p_tstat1 (when -x in command) is the (1-P) result from the GLM parametric analysis?

 

Yes, this is the uncorrected 1-P for the t statistic.

What does out_tstat1 hold?

 

This holds the t statistic, not p-values.

To extract multi-cluster information, I should therefore replace grot_tfce_corrp_tstat1 with grot_tfce_p_tstat1

in the example in the documentation:

"fslmaths grot_tfce_corrp_tstat1 -thr 0.95 -bin -mul grot_tstat1 grot_thresh_tstat1

cluster --in=grot_thresh_tstat1 --thresh=0.0001 --oindex=grot_cluster_index

--olmax=grot_lmax.txt --osize=grot_cluster_size"

 

By doing so you are replacing the corrected for the uncorrected p-values. I think most people are interested in corrected results, so I wouldn't make any change, unless perhaps to explore a bit the results.

In one run there showed

"Warning: All design columns have zero mean - consider using the -D option to demean your data"

Ignoring it seemed to work!

 

You should include the -D, or include a column of ones to account for the intercept. Failing to do so won't give any errors (so, it works), but be aware that the regression line will cross the ordinate (y-axis) at zero, which may be a less-optimal model.

All the best,

Anderson

 

 

Thanks for your patience as I inch forward on this,  Leighton

 

From: FSL - FMRIB's Software Library [mailto:[log in to unmask]] On Behalf Of Anderson M. Winkler
Sent: Wednesday, 1 May 2013 7:59 PM
To: [log in to unmask]
Subject: Re: [FSL] randomise defeated by nuisance covariate(s)?

 

Hi Leighton,

See below:

 

2013/5/1 Barnden, Leighton (Health) <[log in to unmask]>

Thanks Anderson,

 

Didn’t read carefully – as soon as I saw residual I thought ‘noise’ as is usually the case for a full (parametric) GLM analysis. Maybe adding thus might help others

"The images are first fit to the nuisance effects alone and nuisance-only residuals are formed (images with the nuisance effects removed). These images are permuted [...]"

I have run ‘randomise –T’ OK and the 1.0-P results (out_tfce_p_tstat1.nii) look as expected, but the T image (out_tstat1.nii.gz) doesn’t seem to make sense, eg where 1.0-P = 0.998 the same voxel has T = 0.0229. What am I missing here?

You are looking at the p-values computed for the TFCE statistic (produced with the option "-T"), and trying to see if it looks similar to the p-value for the t-statistic in the same voxel. The is no reason to expect them to match because they are different things.

If you'd like to compute voxelwise p-values for the t-statistic, use the option "-x". Even so, the p-value won't be identical to what you would find looking at the standard t-distribution. The reason is that randomise tests are based on permutation, and do not use the t-distribution to compute p-values, even when using t-tests or the t-statistic.

 

All the best,

Anderson


 

Thanks Leighton

____________________________________

 

Hi Leighton,

This excerpt discusses permuting the data, or equivalently, the rows of the design matrix. When there are nuisance variables, this procedure is questionable (in fact, it remains valid for most practical cases, but it's still questionable on theoretical grounds). However, if you continue reading the page you see that randomise deals with covariates using a different method. In the same page you'll read:

"The data are first fit to the nuisance effects alone and nuisance-only residuals are formed. These residuals are permuted [...]"

So it's not the exchangeability of the data or design that matters, but the exchangeability of the residuals once the covariates have been taken into account. So, you are fine with the design that you described as long as the subjects are unrelated to each other and you don't have repeated measurements, as these would require different treatment.

Note however, that you can't compute "TFCE for clusters". Or you compute stats for clusters, or you use TFCE. Can't mix them.

Hope this helps.

All the best,

Anderson

 

2013/4/26 Michael Harms <[log in to unmask]>

HI Leighton,
If you want to understand how randomise works in the presence of nuisance
covariates, I suggest you read the statistics papers cited on the
randomise web page, as it is rather technical, and not easy to explain.

cheers,
-MH

--
Michael Harms, Ph.D.

-----------------------------------------------------------
Conte Center for the Neuroscience of Mental Disorders
Washington University School of Medicine
Department of Psychiatry, Box 8134
660 South Euclid Ave.           Tel: 314-747-6173
St. Louis, MO  63110                    Email: [log in to unmask]




On 4/25/13 6:45 PM, "Leighton Barnden" <[log in to unmask]>
wrote:


>With your help I now have the correct data & covariate input formats for
>randomise. The next hurdle could be trickier though. I want to generate
>TFCE stats for clusters from a simple regression (with severity),
>adjusting for 2 nuisance covariates (global signal level and age). From
>the 'randomise' (theory) wiki :
>
>"Permutation tests rely on an assumption of exchangeability; with the
>models above, the null hypothesis implies complete exchangeability of the
>observations. When there are nuisance effects, however, the null
>hypothesis no longer assures the exchangeability of the data (e.g. even
>when the null hypothesis of no FA difference is true, age effects imply
>that you can't permute the data without altering the structure of the
>data)."
>
>This seems to imply that when the scan order is permuted by randomise the
>order of the nuisance covariates does not follow. If the nuisance
>covariates were always attached to the same scans would not this satisfy
>'complete exchangeability of the observations'? Could this be made an
>option (the default?) for handling of nuisance covariates by randomise?
>Incidentally, inclusion of '--tfce_H 2 --tfce_E 0.5 --tfce_C 6' on the
>randomise command line causes it to fail (FSL5.0).
>Thanks  Leighton