Hi,
...
> First, note that for a GLS mixed-effects analysis (where you weight subject
> i's data and design matrix row by w_i = 1/sqrt(sigma^2_Between +
> sigma^2_{i,Within}) ), and a one-sample 2nd level model (design matrix just
> a column of ones), the estimate of the group mean is
> sum_i w^2_i Y_i / K
> where K = sum_i w^2_i, and Y_i is the contrast estimate for subject i (i.e.
> what you usually take to the 2nd level). (This *not* what is done in SPM,
> but is the standard thing in FSL's Feat).
I wonder, would this still be the maximum likelihood estimator? If one
scales the data, then the log likelihood contains an additional term,
referring to the scaling (this is what one finds in textbooks on such
models, like Pinheiro & Bates 2000, p. 202-203).
In the proposal to take t values to the second level, information on
the scaling is lost (it is a REML without any scaling info, apart from
not even being the GLS model as you note below). I conclude that the
resulting estimator at the second level is unbiased (like all weighted
estimators), but less efficient.
Consider the scenario where I am estimating in a region where the null
is true. Then both the betas and the t's have expected mean zero, but,
as the sample size N -> Inf, var_within(beta) -> 0, while
var_within(t) -> 1. This does not speak for the efficiency of t, even
if the contribution of var_within decreases with increasing N. I
conclude than at least soon after var_within(beta) < 1, t is less
efficient. This seems to confirm my previous conclusion.
Under this argument, is you want to account for heteroscedasticity
among subjects, do not go for t's.
>
> If sigma^2_Between >> sigma^2_{i,Within} then the w_i's are actually all
> about the same, and this GLS will reduce to OLS. Since this is usually
> what we expect (Between>>Within), it partially explains why we don't bother
> with mixed-effects GLS in SPM.
I'd like to present a complementary account of what happens. If I have
the same number of observations in each group, then the loss of
efficiency in ignoring heteroscedasticity in testing the intercept at
the 2nd level is very small. Hence, even in the presence of
heteroscedasticity, my unweighted betas are nearly fully efficient.
The price I pay to get into a GLS mixed level model, in contrast, may
be considerable: I no longer have a simple way of estimating the
small-sample variance of my estimates, so my t's may be off the mark.
Let alone max(t). Fortunately, if Between>>Within, it really does not
matter what I do.
Under this argument, the unweighted betas are almost always best, so
do not go for GLS either.
>
> If sigma^2_Between = 0, then the GLS reduces to a weighted fixed effects
> analysis,
> sum_i Y_i / sigma^2_{i,Within} / K' (*)
> where K' = sum_i 1/sigma^2_{i,Within}. Any mention of "fixed effects"
> sounds awful, but *if* sigma^2_Between = 0 there *isn't* any random effect,
> and this is the correct model to make population inference.
An argument against this would be having a considerably strong prior
against the conclusion that people or brains do not differ among each
other. Most likely, there are not enough data to restrict the
confidence interval on sigma_Within to conclude that it is not zero.
However, this is one of those points where people disagree as a matter
of philosophy (whether or not to simplify the model in such cases).
Even so, the argument here is a brave one in that it pleads for
simplifying on sigma_Between but not on heteroscedasticity of
sigma_Within. The prices for misspecification may differ a lot in
these two cases, and the two are not on the same footing in terms of
the complexity of the model.
Roberto Viviani
Dept. of Psychiatry III
University of Ulm, Germany
>
> Analyzing T's at the 2nd level (for a one-sample model), will give you a
> mean estimate
> sum_i Y_i / sigma_{i,Within} / N (**)
> where N is the number of subjects. Eqn (*) is a weighted mean of effect
> magnitudes, and Eqn (**) is an unweighted mean of test statistics, but
> they're not crazy-different.
>
> What's the point of all this? In Thirion et al. (2007), a whole slew of
> group-level models are evaluated on repeated samples of N=10 subjects (from
> a population of 81 subjects). In their evaluations, Eqn (*) is "Pseudo
> MFX", and it is found to be the most reproducible both in terms of kappa
> and a cluster stability measure (Fig. 9). Crucially, note that all the
> inferences in Thirion et al. are made with permutation, so there's no
> problem with biased standard errors or wrong DF.
>
> So, to the extent that (**) is a weakened version of (*), it might have
> attractive empirical properties, but should be investigated in the settings
> of interest.
>
> -Tom
>
> Thirion, B., Pinel, P., Mériaux, S., Roche, A., Dehaene, S., & Poline,
> J.-B. (2007). Analysis of a large fMRI cohort: Statistical and
> methodological issues for group analyses. *NeuroImage*, *35*(1), 105-20.
> doi:10.1016/j.neuroimage.2006.11.054
>
>
> On Fri, Mar 23, 2012 at 12:29 PM, DRC SPM <[log in to unmask]> wrote:
>
>> Hi all,
>>
>> Another reason for not doing between-subject analysis of
>> within-subject t-statistics is that it would introduce an unwanted
>> dependence on the number of first-level scans for each subject. As the
>> number of scans increases, the beta estimates (just) get more precise
>> but do not tend to larger and larger values, while the t-statistics
>> (for a given beta and sigma) do keep increasing with more scans,
>> proportional to sqrt(n).
>>
>> I don't think the Fisher Z-transformed correlation coefficients would
>> have this problem, but there is a second, perhaps more important,
>> problem, which is that all of t, z, r and various t->z or r->z, are
>> essentially signal-to-noise measures. Usually, the appropriate
>> question at the between-subject level is whether the within-subject
>> signal (itself) is significant compared to the appropriate
>> mixed-effects measure of noise (variability) which is not the same as
>> whether the within-subject signal-to-noise measure is significant
>> compared to the between-subject noise.
>>
>> This problem is not such an issue for the common one-sample t-test at
>> the second level, but can be a major problem for more interesting
>> designs. E.g. consider a regression against age; if you use an SNR
>> measure to summarise the first-level, then a significant
>> slope/correlation at the second-level could be purely driven by higher
>> noise in elderly subjects, rather than a change in activity (or
>> connectivity). The same would be true for a two-sample comparison of
>> young vs. old groups, or control vs. patient groups. If you really
>> want to know where patients activated less or showed weaker
>> connectivity, then you don't want to be able to get significant blobs
>> simply because patients were noisier or moved more or similar.
>>
>> So one answer to the question "why is it okay to compare Z-scores with
>> resting timecourses, but not with task data", is that perhaps it's not
>> okay. It happens to be quite common practice, but that alone doesn't
>> mean that it's correct. In some circumstances, it might be
>> appropriate, but in general, a pure signal measure is probably best
>> (such as beta, either from an activation study or a resting state one,
>> where beta would simply be the slope of the regression of each voxel's
>> activity against the seed).
>>
>> In case this sounds very controversial, I should note that Karl
>> Friston makes essentially the same point here:
>> http://online.liebertpub.com/doi/abs/10.1089/brain.2011.0008
>>
>> Sorry for the long message; I hope it is of some interest. Best wishes,
>>
>> Ged
>>
>>
>> On 21 March 2012 20:31, MCLAREN, Donald <[log in to unmask]> wrote:
>> > A couple of quick comments:
>> > (1) In the resting state, the Z-score is commonly used (although it
>> should
>> > be noted that its not a true Z-score but the Fisher r-to-Z). One could
>> also
>> > compute the Z-score for any task data as well.
>> >
>> > (2) The reason for not using t-statistics is that they are not normally
>> > distributed. If you convert to a Z-equivalent then you can use them.
>> >
>> > (3) The issue you and Chris raised about low, but consistent
>> T-statistics is
>> > an issue with beta estimates as well. It might not be significant in any
>> > subject, but it is consistent, which is interesting.
>> >
>> > (4) In the past, people have used a fixed effects analysis of Z-scores.
>> See
>> > Bosch V. Statistical analysis of multi-subject fMRI data: assessment of
>> > focal activations. JMRI 2000. I haven't seen any arguments to state the
>> this
>> > is a flawed approach - but am open to reading references that make that
>> > claim. Large samples wouldn't need a huge huge effect. Also, this ignores
>> > the between subject variance.
>> >
>> > (5) The question is how to interpret low amplitude or low fits in group
>> > studies is important, but I don't know of a good answer. A related
>> question
>> > might be why is it okay to compare Z-scores with resting timecourses, but
>> > not with task data?
>> >
>> > (6) I'll also suggest an alternative approach. One could threshold each
>> > subject's first level map and code the significant voxels with a value
>> of 1.
>> > Then create a map that shows the number or percentage of subjects that
>> have
>> > significant activation or deactivation at a particular voxel.
>> >
>> >
>> > Best Regards, Donald McLaren
>> > =================
>> > D.G. McLaren, Ph.D.
>> > Postdoctoral Research Fellow, GRECC, Bedford VA
>> > Research Fellow, Department of Neurology, Massachusetts General Hospital
>> and
>> > Harvard Medical School
>> > Website: http://www.martinos.org/~mclaren
>> > Office: (773) 406-2464
>> > =====================
>> > This e-mail contains CONFIDENTIAL INFORMATION which may contain PROTECTED
>> > HEALTHCARE INFORMATION and may also be LEGALLY PRIVILEGED and which is
>> > intended only for the use of the individual or entity named above. If the
>> > reader of the e-mail is not the intended recipient or the employee or
>> agent
>> > responsible for delivering it to the intended recipient, you are hereby
>> > notified that you are in possession of confidential and privileged
>> > information. Any unauthorized use, disclosure, copying or the taking of
>> any
>> > action in reliance on the contents of this information is strictly
>> > prohibited and may be unlawful. If you have received this e-mail
>> > unintentionally, please immediately notify the sender via telephone at
>> (773)
>> > 406-2464 or email.
>> >
>> >
>> >
>> >
>> > On Wed, Mar 21, 2012 at 11:10 AM, Jonathan Peelle <[log in to unmask]>
>> wrote:
>> >>
>> >> Hi all,
>> >>
>> >> From time to time the question comes up regarding performing
>> second-level
>> >> (group) analyses on contrast images vs. t statistics obtained from
>> >> first-level (single-subject) analyses. The conventional wisdom is that
>> >> performing second-level statistics on the con* images is more
>> appropriate.
>> >> This makes sense to me, as the contrast images reflect effect size, and
>> thus
>> >> are testing whether the effect differs from 0 across the group.
>> >>
>> >> However, is it technically inappropriate to use t statistics (or their Z
>> >> equivalent) for second-level analysis? What is the rationale either
>> way? And
>> >> in particular, if it's not inappropriate, what would the interpretation
>> be?
>> >>
>> >> One challenge that comes to mind is the interpretation of a non-zero
>> >> effect. For example, a group of subjects may all have a t statistic of
>> 0.1.
>> >> This is consistently greater than 0 (which is what a second-level
>> one-sample
>> >> t-test would show), but none of us would consider a t value of 0.1
>> >> particularly meaningful. This is in contrast to a parameter estimate
>> >> differing from 0, which is easily interpreted as there being a
>> significant
>> >> effect across subjects.
>> >>
>> >> This is obviously not an issue restricted to neuroimaging, but thus far
>> >> I've not found a discussion of the topic in any context. Any opinions
>> would
>> >> be most welcome (as would relevant references)!
>> >>
>> >> Best regards,
>> >>
>> >> Jonathan
>> >>
>> >> --
>> >> Dr. Jonathan Peelle
>> >> Center for Cognitive Neuroscience and
>> >> Department of Neurology
>> >> University of Pennsylvania
>> >> 3 West Gates
>> >> 3400 Spruce Street
>> >> Philadelphia, PA 19104
>> >> USA
>> >> http://jonathanpeelle.net/
>> >
>> >
>>
>
>
>
> --
> __________________________________________________________
> Thomas Nichols, PhD
> Principal Research Fellow, Head of Neuroimaging Statistics
> Department of Statistics & Warwick Manufacturing Group
> University of Warwick, Coventry CV4 7AL, United Kingdom
>
> Web: http://go.warwick.ac.uk/tenichols
> Email: [log in to unmask]
> Phone, Stats: +44 24761 51086, WMG: +44 24761 50752
> Fax: +44 24 7652 4532
>
|
