Dear Kewei,
> Our group is interested in determining the extent to which results
> from previous exploratory PET studies of hunger, satiation and taste
> (n=11) is reproducible in an independent group of 11 subjects who were
> studied using the identical image-acquisition and image-analysis
> procedures.
>
> Before we analyze the second data set, we wanted to have clear
> criteria for replication. We were thinking of generating SPM's in the
> second data set, and over-lapping the results of the second study
> with those from the first. Those areas associated with significant
> activations in both data sets would be considered replicated
> findings. We don't think we can say much about those regions
> associated with increased activity in only one of the two studies
> other than that we failed to replicate these findings, since we
> probably don't have the statistical power to refute changes in these
> regions.
>
> Does that make sense to you? What have been the criteria used in
> previous studies to confirm previous findings from the same
> laboratory? What would be the criteria to disconfirm a previous
> finding given our limiations in statistical power? Since we have
> replication data--a nice way to address the problem of multiple
> comparisons, how liberal can we logically make the statistical
> threshold for each of the statistal maps (e.g., .05,.01, .005,
> etc)? And does it make sense not to make too much of any negative
> findings (given limitations in statistical power), or should we
> consider additional means to say with some power that a particular
> finding was disconfirmed?
Your proposal relates to the notion of a split T-test and through that,
formally, to a conjunction analysis. One simple way to approach this
issue is to combine both studies in a single (2-study) statistical
model and perfom a conjunction analysis of the same contrasts for each
study. This will identify regions that activate in both studies and
also harnesses the implicit reduction in search volume (each study is
effectively masked by the other). Generally, people do not split their
data because (according the the Gauss-Markov theorem) the minimum
variance, or most efficient, estimator of the activation derives from
pooling all the data into one analysis. However split T-tests can be
more robust under some violations and serve the special purpose of
addressing reproducibility in an explicit fashion. In fMRI
conjunctions are used to perform split T-tests to confirm that every
subject activated in multi-subject studies and this represents a
standard approach you could adopt.
I hope this helps - Karl
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
|