This is a disadvantage of FIR; there's no obvious "most canonical" way to measure effect size.
You don't really have any alternative other than to decide on a measure of effect size and then do statistical inference on it. One measure was my suggestion, summing/averaging over time bins. I can understand that it might not be right for your particular situation, but you have to come up with some measure of "effect size" that's right for you and makes sense, and for which you know how to come up with a measure of statistical significance.
The problem isn't really statistical, but simple math. If I show you two graphs from, say, x = 0 to x = 20, which graph is "bigger"? The one which has a greater maximum? The one with the greater average, or area underneath? There's no single best answer.
I'm not sure which F-test you used, but it would probably measure a condition-by-time-bin interaction. It's asking the question, across all the conditions (or perhaps between two conditions; or perhaps one condition and baseline...again, not sure about the specific F-test you used), is there a significant difference between the conditions in one or more of the time bins?
In terms of a summary statistic that doesn't allow positive and negative to cancel each other, and so on, you could use anything you like (e.g., sum of absolute values), but then you have to figure out what the null hypothesis distribution looks like. (Meaning, the sum of absolute values of things that behave "normally" (according to Gaussian stats) will no longer be Gaussian.) That stuff can get pretty esoteric very quickly.
Stephen J. Fromm, PhD
Contractor, NIMH/MAP
(301) 451--9265
________________________________________
From: Dirk den Ouden [[log in to unmask]]
Sent: Monday, September 28, 2009 10:49 AM
To: [log in to unmask]; Fromm, Stephen (NIH/NIMH) [C]; [log in to unmask]
Cc: [log in to unmask]
Subject: Re: [SPM] subtracting and thresholding spmF maps
Thanks Donald, Michael and Stephen,
one of the issues is that I want to do this in single-subject analyses, so I believe the flexible
factorial, which I agree would otherwise be the way to go, is not possible here (with only one scan
per cell), and I think a one-sample t-test over difference images isn't either.
I would indeed be looking for a summary response for the FIR, but I don't think I should be simply
averaging over the different time bins, or summing them; the 'response' being mostly in how the
time bins differ from each other, rather than their averaged or summed amplitude (which may in fact
be 0 if the response shows undershoots). I thought that was what the F-value represented, viz. a
derived statistic representing 'effect size' ...
If not that, what *does* the F-value represent, and how is it computed (and how can I compare it
across conditions/runs in a single subject?)?
Dirk
On Mon, 28 Sep 2009 9:23:21 am CDT "MCLAREN, Donald" wrote:
Another approach would be to use the flexible factorial design to setup a
repeated-measures analyses. You would have three factors: subject,
condition, and time. You would also include time*condition interaction. The
input images would be the beta images for each time bin of the FIR.
What you will want to look at is the interaction term -- where the
conditions vary differently over time. You can also look at the main effect
of condition as well.
On Mon, 28 Sep 2009 8:26:27 am CDT "Stephen J. Fromm" wrote:
You definitely should _not_ do "statistics on significance" by computing the
significance of differences in the F-statistics.
Instead, you should devise a summary measure for the FIR, such as "sum of
responses for time bins 3 through 7".
On Fri, 25 Sep 2009 6:39:57 pm CDT Michael T Rubens wrote:
I believe you'd contrast the ess files (subtract a-b), then do a t-test on
the result.
hth,
Michael
On Fri, Sep 25, 2009 at 12:42 PM, Dirk den Ouden
<[log in to unmask]>wrote:
> Dear all,
>
> within single subjects, I want to compare effect sizes for different
> conditions and between time points, irrespective of the shape of the hrf
> (even negative responses and weird shapes are allowed), as this may be
> different between regions and change over time in the subjects I am
> studying. I have modelled the responses to the various conditions as FIR
> functions (10 time bins of 2 seconds) and main effects are easily calculated
> that way, with an F-test. However, statistical comparison between conditions
> is trickier, without assuming a specific shape or even time-to-peak of the
> responses. NB, I do not want to compare specific time bins to each other -
> all I want is to compare the effect sizes of the overall responses,
> irrespective of shape.
>
> It seems to me I can subtract the spmF_000x.img files for 2 conditions, the
> resulting image giving me the difference in F-values (representing effect
> size) between these two conditions, for each voxel. However, I am not sure
> what height threshold to use on this difference-image. Should I use the
> height threshold, in terms of an F-value, that I obtain from performing a
> main-effect analysis for 1 of the conditions, with a specific correction for
> multiple comparisons, or is there another calculation that will tell me what
> height threshold to use if I want to report, for example, only voxels with a
> significantly different effect size at an FWE corrected alpha level of .05?
>
> Any comments or advice is much appreciated!
> Thanks,
> Dirk
>
>
> ************************************
>
> Dirk-Bart den Ouden, Ph.D.
>
>
>
> Aphasia & Neurolinguistics Research Laboratory
>
> Dept. of Communication Sciences & Disorders
>
> Northwestern University
>
> 2240 Campus Drive, Evanston IL 60208-3066
>
> Phone: (office) 1-847-467-2515; (lab) 1-847-467-7591
>
> ************************************
>
--
Best Regards, Donald McLaren
=====================
Support the Alzheimer's Association Memory Walk 2009 ~~~
Join the Wisconsin Alzheimer's Disease Research Center team or make a
donation.
http://madison.kintera.org/2009/donaldmclaren
=================
D.G. McLaren
University of Wisconsin - Madison
Neuroscience Training Program
Office: (608) 265-9672
Lab: (608) 256-1901 ext 12914
=====================
This e-mail contains CONFIDENTIAL INFORMATION which may contain PROTECTED
HEALTHCARE INFORMATION and may also be LEGALLY PRIVILEGED and which is
intended only for the use of the individual or entity named above. If the
reader of the e-mail is not the intended recipient or the employee or agent
responsible for delivering it to the intended recipient, you are hereby
notified that you are in possession of confidential and privileged
information. Any unauthorized use, disclosure, copying or the taking of any
action in reliance on the contents of this information is strictly
prohibited and may be unlawful. If you have received this e-mail
unintentionally, please immediately notify the sender via telephone at (608)
265-9672 or email.
===========End of original message text===========
|
