Dear Karl,
Thank you very much for helping to clear up confusion with regard to what
effect sizes are being plotted in SPM and in several of your group's
research articles. I just needed to be sure I wasn't missing something in
how this was being presented.
Also, thanks for your comments with regard to my questions about how best
to understand and report effect size estimates. Eric Zarahn brings up
several additional points that are very relevant to this whole issue and
probably have not been fully appreciated. Another might be the small
subject sample sizes in many studies. Considering the recent interest in
population-level generalization of neuroimaging results, how statistical
effect sizes are represented and how these other issues contribute to
effect size take on an increased importance. Again, I look forward to
hearing what others have to say about these issues.
Warmest regards,
Jack
-----Original Message-----
From: Karl Friston [SMTP:[log in to unmask]]
Sent: Thursday, April 08, 1999 12:48 PM
To: [log in to unmask]
Cc: [log in to unmask]
Subject: Re: *Statistical Effect Size*
Dear Jack
> I have a couple of questions about how statistical effect size is being
> represented in SPM as well as in several recent articles from your
> group. Perhaps you can clarify this for me and also address a larger
issue
> related to the reporting of statistical effect size.
>
> In your recent manuscript ("Multi-subject fMRI studies and conjunction
> analyses") Figures 3 and 4 have Y-axes labels indicating "Effect Size"
> but it is unclear what this measure of effect is with respect to. Are
> these values given in percent change relative to whole brain mean or
> the specific voxel mean?
Whole brain mean: They are simply the fitted and adjusted data that
ensues from the least squares fit. By virtue of global normalization
the units are percentage whole brain mean.
> Additionally, in your email to Dr. Muftuler,
> you indicate that in the paper "Detecting activations in PET and fMRI:
> Levels of inference and power", Neuroimage 40, 223-235 (1996) that the
> 'component score scale' estimates are completely independent of the
> inference (that is the test statisitic or p value) and that if the
> global normalization was to a grand mean of 100 then the scale is
> simply percent signal change relative to the whole brain mean signal.
> So then are these two things actually the same thing - just named
> differently? How do these values relate to other measures of effect
> size that have been reported such as in Buchel et al. (Science, 283,
> pp. 1538-1541, Figure 2)? Is this again another name for the same
> thing?
Yes absolutely. I am using the term 'effect size' in reference to the
'size of the effect' (i.e. a measure of the modeled effect that
reflects the parameter estimate as opposed to the statistic that is
used for inference.
> In a more philosophical vein, can percent change indices really be
> considered measures of effect size in the traditional sense? For
> instance, classical effect size estimates are indeed 'adimensional',
> based upon the inferential test statistic used to evaluate the
> statistical model, take into account measurement variance, and are
> independent of the study sample size. They are typically taken to be
> population level estimates and include such measures as Cohen's d
> statistic (for evaluating t-tests), eta-squared (for ANOVA, etc), among
> others. These measures are often used to make estimates of the number
> of subjects needed to reliably obtain a statistically significant
> experimental result. Additionally, statistical test values may be
> culled from research articles in the literature, converted to effect
> size estimates, and assessed under meta-analysis, or as been more
> recently discussed and demonstrated, pooled across individual subjects
> in an fMRI investigation to provide evidence for consistency of
> activation over a subject sample (e.g. sum(Z)/sqrt(N), Chi-square, or
> your conjunction analysis approach). However, the percent change
> measure is not a standardized difference with respect to the variation
> in the measurement but is difference taken as a ratio to the whole
> brain mean or the mean of some other stimulus condition. Since the
> method for computation of percent change appears to vary between
> reports in the literature as well as experimental designs, are there
> any troublesome distributional properties to worry about that might
> express themselves if this measure is looked at in meta-analysis? Are
> there any advantages to reporting effect sizes like the classical
> measures mentioned above in order to facilite better comparison between
> studies, permit better estimates of statistical power, as well as
> enable evaluation of the overall body of research? Could you comment on
> this?
(i) You clearly have an understanding of 'effect size in the
traditional sense' that far surpasses mine. I repeat our use of the
term is probably improper and refers to the size of the modeled effect,
(as opposed to its significance i.e. size corrected for error variance
and errors in the estimation of that error variance).
(ii) I think it is a very interesting idea to have these sorts of
summary statistics in relation to activation data. It is not something
that we have considered carefully. I look forward to hearing other
comments.
> I appreciate your clarifying these issues and perhaps there are others
> on the list can provide some additional thoughts on the broader topic
> how to best report statistical effect size from neuroimaging studies.
Very best wishes - Karl
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
|