Hello Jiansong,


Quoting Jiansong Xu <[log in to unmask]>:


> Assume I scanned 100 subjects, and after the analysis of the
> first 50
> subjects, I found a cluster of 100 voxels in the midbrain pass
> the threshold
> of p < 0.01. Then I definde this cluster as a ROI.  If the next
> 50 subjects
> also showed a cluster in the same region, and survived the SVC of
> the ROI
> definded by the first 50 subjects, then can I say this cluster is
> really
> statistically significant?

Assuming the 100 subjects were independently sampled from the
population of interest (which seems a reasonable assumption in most
cases), yes the p-value for the second sample's analysis would be
valid. This is because the SVC test was not conditioned on those
data.

Eric





>
> Jiansong
>
>
>
> > From: Rajeev Raizada <[log in to unmask]>
> > Reply-To: Rajeev Raizada <[log in to unmask]>
> > Date: Wed, 29 Aug 2007 19:35:32 -0400
> > To: <[log in to unmask]>
> > Subject: [SPM] Q: Objective criteria for when small-volume
> correction is
> > valid? [Re: Q: Are small midbrain nuclei unfairly penalised by
> > multiple-comparison correction procedures?]
> >
> > On Wed, 29 Aug 2007, Eric Zarahn wrote:
> >
> >> Hello,
> >>
> >> Coming in late to this thread, but just wanted to comment:
> >>
> >>>> Suppose you find an activation cluster that you believe may
> originate
> >>>> from a small midbrain nucleus, and you want to test whether
> it is
> >>>> significant at a p-value that is properly corrected for
> multiple
> >>>> comparisons.
> >>>>
> >>>> One way would be to contrive a small-volume correction,
> after dredging
> >>>> PubMed to find suitable previous papers to justify your ROI.
> >>
> >> Such a procedure would not control the p-value at the nominal
> SVC level
> >> as the ROI selection for use in a particular dataset was based
> on seeing
> >> a cluster in that ROI in that dataset (what other evidence one
> might
> >> find after the fact, on say PubMed or anywhere else, would be
> irrelevant
> >> in terms of the p-value associated with this procedure).
> >>
> >> Eric
> >
> > Many thanks for the comment.
> > If I understand you correctly, what you're saying essentially
> is that
> > a small-volume correction is valid only if the small-volume
> that you pick
> > really truly did arise from a prior hypothesis that you thought
> up
> > beforehand.
> >
> > This highlights an aspect of small-volume correction that
> puzzles me.
> > Naturally, we all as researchers have all sorts of expectations
> and
> > hunches beforehand about what our dataset might show.
> > In that sense, the "dredging PubMed" line was a bit of poetic
> license.
> > Indeed, it's usually only the ROIs that make sense in light of
> our
> > prior hunches that we choose to investigate further.
> > That's true for this possible midbrain nucleus, as well.
> >
> > Does this mean that the statistical validity of a small-volume
> correction
> > hinges upon the externally unverifiable factor of whether
> > a researcher's prior hunch about possible activations
> > was really truly felt beforehand. Or whether they wrote it
> down,
> > and then signed and dated the paper?
> >
> > It doesn't seem correct that something as concrete as the
> validity
> > of a corrected p-value should hinge on the temporal ordering
> > of thoughts that happened to pass through a researcher's mind.
> >
> > Put another way: nobody ever runs a small-volume correction
> > on regions of the T-map that look as if they are probably
> empty.
> > But if the validity of the procedure hinges upon pre-stating
> > hypotheses and then acting on them regardless of how the data
> look,
> > then that is what people should be doing, no?
> >
> > Is there not some objective set of criteria for determining
> > when it is, and when it is not, valid to run a small-volume
> correction?
> >
> > Many thanks,
> >
> > Raj
>