Tom,

I have been following this post and have a couple related questions. 
Regarding cluster mass and cluster size thresholds, what kind of criteria 
can you apply to say that the thresholding worked "well"? For instance, how 
do you judge that cluster mass works better than cluster size? It seems 
that it is fairly undefinable which is why TFCE is usually recommended. But 
if there are certain cases when an a priori cluster mass threshold would be 
appropriate (and how to determine when that is and what the threshold might 
be) as opposed to using TFCE, I would be happy to know what they are, 
otherwise I will gladly stick with TFCE. The less decisions I have to make 
the better:)

Also regarding trend-level results using TFCE, do you think that voxels 
with p< .1 or even p < .2 can be "hypothesis generating" or rather would 
you dismiss these as noise? You commments seem to suggest that even in 
complete noise you are likely to find near significant results, so with the 
TFCE method perhaps it is best to hold solid at p < .05? "you'll almost 
always find
some extreme t-values or nearly significant clusters". It is also 
interesting that in practice even using TFCE, I tend to apply an informal 
cluster-threshold. Often one or two voxels will pass significance at p < 
.05, but I usually believe them to be noise, b/c lowering the threshold 
does not tend to make larger clusters, so I just usually believe these are 
one/two voxels of extreme noise, even though technically they have passed 
TFCE. Thanks for any guidance, it is always very helpful.

Chris Bell
University of Minnesota



On Oct 5 2010, Thomas Nichols wrote:

>Jay,
>
 I'm afraid you've simply got a very fiddly result. For an *a priroi* 
choice
>of cluster-forming threshold, I'm quite fond of cluster mass as it seems to
>do as well or better than cluster size.  But cluster size and cluster mass
>are both sensitive to the exact cluster-forming threshold, which motivated
>the development of TFCE.  In my limited experience, a result found with
>cluster size/mass but not with TFCE is one that is very
>cluster-forming-threshold-dependent... if you change your threshold up or
>down from 4 I bet you'll lose the interesting clusters you mention.
>
>Sorry I don't have a better suggestion... if the 4 threshold with cluster
>mass was the first thing you tried, then it's a good result.  But if it was
>found after various attempts at different cluster-forming thresholds, I
>won't trust it.
>
>-Tom
>
>On Tue, Oct 5, 2010 at 11:11 AM, Jay Ives <[log in to unmask]> wrote:
>
>>  Hi Tom,
>>
>> Thanks for the (bad) news, although I am grateful for the reality check.
>>
   The reason I'm pursuing this is that I get interesting significant 
clusters
   if I use cluster extent or mass with a t threshold of 4, but these 
clusters
>> do not reach significance with tfce. I've tried several variations of the
>> tfce H and E values, without success.
>>
>> The other clusters in quite separate part of the brain show similar t
>> values to the ones I am trying to "massage" into significance.
>>
>> I'm confused as to what to do...especially as I am getting different
>> results with tfce and cluster extent or mass.
>>
>> If you are willing, I can provide my images for your comments.
>>
>> Thanks again.....J
>>
>> ----- Original Message -----
>> *From:* Thomas Nichols <[log in to unmask]>
>> *To:* [log in to unmask]
>> *Sent:* Tuesday, October 05, 2010 5:25 PM
>> *Subject:* Re: [FSL] Use of masks in randomise for VBM
>>
>> Dear Jay,
>>
>> I wanted to jump in on this thread...
>>
>>   Is it also a "bad idea" to re-run randomise with a more localised
    anatomical mask (say for example the posterior half of the brain) once 
one
>>> has found a nearly significant cluster in the occipital lobe?
>>>
>>
>> Yes, this is the same bad idea.  If you want to be convinced, just try
>> taking your data and 'messing' up the group labels, manually creating one
   permutation; look at the statistic image, and you'll almost always find 
some
   extreme t-values or nearly significant clusters, even though you know 
you're
   staring at junk. If the issue of circularity isn't clear, see this 
paper:
>>
>>   Kriegeskorte, N., Simmons, W., Bellgowan, P., & Baker, C. (2009).
>> Circular analysis in systems neuroscience: the dangers of double dipping.
>> *Nature Neuroscience*, *12*(5), 535–540.
>>
>>
>>  Out of interest, does a significant cluster in the anterior half of the
    brain alter the potential significance of a nearly significant cluster 
in
>>> the posterior half?
>>>
>>
   With randomise it is theoretically possible that a *huge* signal in part 
of
>> the brain could deflate the FWE significance of results elsewhere.  While
   we've seen this effect in simulations (where gigantic, high SNR signals 
were
   added), we never could see the effect in simulations with moderate SNR's 
or
>> in real data (actual signals are just too subtle).  If you truly think
>> you're affected by this, let us know.
>>
>> -Tom
>>
>>
>>
>>>  Thanks again......J
>>>
>>>  ----- Original Message -----
>>> *From:* Matthew Webster <[log in to unmask]>
>>> *To:* [log in to unmask]
>>>   *Sent:* Monday, October 04, 2010 8:33 PM
>>> *Subject:* Re: [FSL] Use of masks in randomise for VBM
>>>
>>> Hello Jay,
>>>                   The parameters of the method you use, whether an
>>> anatomical mask or a threshold, should be obtained independently of the
>>> initial randomise results. Running randomise, making a mask of a cluster
    that just isn't significant enough, and then re-running that contrast 
with
    the mask is a very bad idea ( see Re: Using mask of uncorrected output 
in
    TBSS <https://www.jiscmail.ac.uk/cgi-bin/webadmin?A2=FSL;8b4820bf.0709> 
for
>>> more details )
>>>
>>> Many Regards
>>>
>>> Matthew
>>>
>>>  Hi Matthew,
>>>
    I presume you are implying that masking according to uncorrected p 
values
>>> is not a priori?
>>>
>>> What if I use a more localised anatomical mask to improve significance
>>> after finding regions of near significance? Is that OK?
>>>
>>> Thx.....J
>>>
>>> ----- Original Message -----
>>> *From:* Matthew Webster <[log in to unmask]>
>>> *To:* [log in to unmask]
>>> *Sent:* Monday, October 04, 2010 5:43 PM
>>> *Subject:* Re: [FSL] Use of masks in randomise for VBM
>>>
>>> Hello,
>>>           As long as your anatomical mask was derived a priori it should
    be OK. The -T option will generate both uncorrected and corrected 
p-values
    using TFCE, this does not involve a hard threshold ( the --tfce_H 
--tfce_E
>>>  and --tfce_C options can be used to change the algorithm parameters ).
>>>
>>> Many Regards
>>>
>>> Matthew
>>>
>>>
>>>  Hi,
>>>
>>> Thanks, but I don't fully understand your advice.
>>>
    I have used a portion of the provided GM_mask after cropping to an area 
of
>>> interest with '"fslmaths -roi", and found significance increased for the
>>> clusters which were present previously. I presume this is the expected
>>> result, and is OK.
>>>
>>> Would you please also comment on the method used in the paper I read "in
>>> which the authors (using SPM) thresholded images to voxels with t values
    giving uncorrected p < 0.001. The probability of the remaining clusters 
was
>>> then corrected for multiple comparisons".
>>>
    I'm not sure what is going on within "randomise". I see that 
uncorrected p
>>> value images are output (at least with the -T option). Are these images
>>> thresholded before correction for multiple comparisons, as above? Is it
>>> possible to adjust the threshold with an option?
>>>
>>> Thanks
>>>
>>> J
>>>
>>>
>>>
>>> ----- Original Message -----
>>> *From:* Stephen Smith <[log in to unmask]>
>>> *To:* [log in to unmask]
>>> *Sent:* Sunday, October 03, 2010 11:32 PM
>>> *Subject:* Re: [FSL] Use of masks in randomise for VBM
>>>
    Hi - for randomise masking, I would recommend that if you're not using 
the
    gm_mask already generated, you only take *subsets* of that mask if you 
want
>>> to make your own, i.e. don't look *outside* this mask.
>>>
>>> Cheers.
>>>
>>>
>>>  On 2 Oct 2010, at 10:25, Jay Ives wrote:
>>>
>>>  Hi,
>>>
>>> Is it valid to use a roi mask rather than the gm_mask created by the
>>> scripts?
>>> What limitations are there on the choice of the mask?
>>> I have read a published article in which the authors (using SPM)
    thresholded images to voxels with t values giving uncorrected p < 
0.001. The
>>> probability of the remaining clusters was then corrected for multiple
>>> comparisons.
>>> Is this OK, or is one limited to using anatomic roi masks?
>>>
>>> Thx.........J
>>>
>>>
>>>
>>>
     
---------------------------------------------------------------------------
>>> Stephen M. Smith, Professor of Biomedical Engineering
>>> Associate Director,  Oxford University FMRIB Centre
>>>
>>> FMRIB, JR Hospital, Headington, Oxford  OX3 9DU, UK
>>> +44 (0) 1865 222726  (fax 222717)
>>> [log in to unmask]    http://www.fmrib.ox.ac.uk/~steve
>>>
     
---------------------------------------------------------------------------
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>
>>
>> --
>> ____________________________________________
>> Thomas Nichols, PhD
>> Principal Research Fellow, Head of Neuroimaging Statistics
>> Department of Statistics & Warwick Manufacturing Group
>> University of Warwick
>> Coventry  CV4 7AL
>> United Kingdom
>>
>> Email: [log in to unmask]
>> Phone, Stats: +44 24761 51086, WMG: +44 24761 50752
>> Fax:  +44 24 7652 4532
>>
>>
>
>
>