Dear Group
Are there different guidelines for thresholding group data
when looking at random or fixed effects analyses and cluster thresholding?
I have noticed much less robust results (z-scores) with
random effects (for areas with less signal to noise such as
the amygdala). p<0.001 seems excessive
Thanks a lot
Jeff Lorberbaum
On Thu, 3 Mar 2005, Russ Poldrack wrote:
> Mikkel - this is generally a problem with classical hypothesis testing: since
> the point null hypothesis is never true, if you collect more data you will
> always reject it. It's a good argument IMHO for Bayesian inference methods,
> where the inference becomes more precise as you collect more data. However,
> this isn't really any issue for most fmri studies using random effects
> analyses, as we never have enough power to find insubstantial effects (if
> anything the worry is that we are generally under-powered).
>
> cheers
> russ
>
> On Mar 3, 2005, at 2:15 AM, Mikkel Wallentin wrote:
>
>> Just a small question from an amateur in relation to reporting
>> unthresholded
>> maps:
>>
>> Isn't it true for fMRI, that the more data you acquire, the larger your
>> blobs become, regardless of anything else? If you just have enough data,
>> you
>> can fit any model and get results.
>> Therefore, isn't it a problem with unthresholded maps that the more power
>> you
>> have, the less meaningful the unthresholded map becomes?
>>
>>
>> Cheers,
>> Mikkel Wallentin
>>
>> ----- Original Message -----
>> From: "Matthew Brett" <[log in to unmask]>
>> To: <[log in to unmask]>
>> Sent: Wednesday, March 02, 2005 8:05 PM
>> Subject: Re: [SPM] Any Papers on Presenting fMRI Results?
>>
>>
>>> Dear Daniel, Mauro,
>>>
>>> Sorry to reply to you both, but I was finding some overlap in what I
>>> wanted to say.
>>>
>>> Thanks again for replies, which were thought-provoking. Here were the
>>> provoked thoughts!
>>>
>>> Daniel wrote:
>>>> I guess I don't think it's fair to expect articles to explicitly
>>>> describe what inferences can't be made from the data. I'm happy with
>>>> just, "area A was significantly more active during A than B."
>>>
>>> which I'm going to claim is kind of the same thing as Mauro wrote:
>>>
>>>> >A passes significance at p=0.05, B doesn't p=0.04. It could very
>>>> >easily be that B has even has a higher effect size than A. It seems
>>>> >to me very misleading to report 'A is significant' without 'B is
>>>> >very close to A'.
>>>>
>>>> Sure, but in such a case, I wouldn't accept any inference about "A
>>>> vs. B" without a specific test. Which brings us back to square one:
>>>> How can we assess differences across areas rather (or in addition to)
>>>> differences across conditions/design?
>>>
>>> The key point here is that I think people _are_ universally drawing an
>>> _implicit_ conclusion about A vs B when commenting on a thresholded
>>> map.
>>>
>>> To take the behavioral example. Let us say you are doing a study on
>>> patients with dorsolateral prefrontal cortex damage and test them on
>>> (task A) spatial working memory and (task B) a stroop task. A gives
>>> p=0.05, B gives p=0.06. You don't report the result for B atall and
>>> only report A, and say, 'frontal lobe patients are impaired on spatial
>>> working memory'. It would be true to say this, but it would be very
>>> misleading, because it implies that patients with frontal lobe lesions
>>> are _particulary_ impaired on spatial working memory, for which you
>>> have no good evidence. The reason that 'frontal lobe patients are
>>> impaired on spatial working memory' implies the unsupported 'frontal
>>> lobe patients are _particularly_ impaired on spatial working memory'
>>> is that, if frontal lobe patients are impaired on all tests, or even
>>> all tests of memory, stating that they are impaired on spatial working
>>> memory is entirely uninteresting.
>>>
>>> Obviously I'm drawing a parellel with the thresholded SPM map. Again
>>> we have done many measurements. Again we are simply not reporting the
>>> results of the large majority of the measurements. Let's say 'Area X
>>> is activated by task A'. On its own, this is misleading, because this
>>> statement would be entirely uninteresting if it is also true that the
>>> whole of the rest of the brain is activated to a similar extent. So,
>>> I believe that 'Area X is activated by task A' actually strongly
>>> implies 'Area X _in particular_ is activated by task A' for which it
>>> is very rare to present any good evidence.
>>>
>>>> One thing we haven't talked about is the kinds of invalid inferences
>>>> encouraged by unthresholded maps. If you have maps from under-powered
>>>> studies of two tasks (B-A and C-A), side-by-side comparison is liable
>>>> to suggest some obvious but false differences and/or similarities.
>>>
>>> Again, this is an important point. Should you remove a lot of your
>>> data by using a thresholded map, and prevent people from drawing
>>> possibly invalid conclusions about the data that is not significant?
>>> My own view would be you should not, and that I would be happy for
>>> someone to make a reasoned argument about - say - an area that was not
>>> significant, but that was close to signficance, looked as though it
>>> was specifically activated (red surrounded by blue) and was bilateral.
>>> That also happens in the behavioral literature - you can discuss
>>> trends in data.
>>>
>>> See you,
>>>
>>> Matthew
>>>
>>>
> ---
> Russell A. Poldrack, Ph.d.
> Assistant Professor
> UCLA Department of Psychology
> Franz Hall, Box 951563
> Los Angeles, CA 90095-1563
>
> phone: 310-794-1224
> fax: 310-206-5895
> email: [log in to unmask]
> web: www.poldracklab.org
>
|