I'm not an expert at all.
Assume I scanned 100 subjects, and after the analysis of the first 50
subjects, I found a cluster of 100 voxels in the midbrain pass the threshold
of p < 0.01. Then I definde this cluster as a ROI. If the next 50 subjects
also showed a cluster in the same region, and survived the SVC of the ROI
definded by the first 50 subjects, then can I say this cluster is really
statistically significant?
Jiansong
> From: Rajeev Raizada <[log in to unmask]>
> Reply-To: Rajeev Raizada <[log in to unmask]>
> Date: Wed, 29 Aug 2007 19:35:32 -0400
> To: <[log in to unmask]>
> Subject: [SPM] Q: Objective criteria for when small-volume correction is
> valid? [Re: Q: Are small midbrain nuclei unfairly penalised by
> multiple-comparison correction procedures?]
>
> On Wed, 29 Aug 2007, Eric Zarahn wrote:
>
>> Hello,
>>
>> Coming in late to this thread, but just wanted to comment:
>>
>>>> Suppose you find an activation cluster that you believe may originate
>>>> from a small midbrain nucleus, and you want to test whether it is
>>>> significant at a p-value that is properly corrected for multiple
>>>> comparisons.
>>>>
>>>> One way would be to contrive a small-volume correction, after dredging
>>>> PubMed to find suitable previous papers to justify your ROI.
>>
>> Such a procedure would not control the p-value at the nominal SVC level
>> as the ROI selection for use in a particular dataset was based on seeing
>> a cluster in that ROI in that dataset (what other evidence one might
>> find after the fact, on say PubMed or anywhere else, would be irrelevant
>> in terms of the p-value associated with this procedure).
>>
>> Eric
>
> Many thanks for the comment.
> If I understand you correctly, what you're saying essentially is that
> a small-volume correction is valid only if the small-volume that you pick
> really truly did arise from a prior hypothesis that you thought up
> beforehand.
>
> This highlights an aspect of small-volume correction that puzzles me.
> Naturally, we all as researchers have all sorts of expectations and
> hunches beforehand about what our dataset might show.
> In that sense, the "dredging PubMed" line was a bit of poetic license.
> Indeed, it's usually only the ROIs that make sense in light of our
> prior hunches that we choose to investigate further.
> That's true for this possible midbrain nucleus, as well.
>
> Does this mean that the statistical validity of a small-volume correction
> hinges upon the externally unverifiable factor of whether
> a researcher's prior hunch about possible activations
> was really truly felt beforehand. Or whether they wrote it down,
> and then signed and dated the paper?
>
> It doesn't seem correct that something as concrete as the validity
> of a corrected p-value should hinge on the temporal ordering
> of thoughts that happened to pass through a researcher's mind.
>
> Put another way: nobody ever runs a small-volume correction
> on regions of the T-map that look as if they are probably empty.
> But if the validity of the procedure hinges upon pre-stating
> hypotheses and then acting on them regardless of how the data look,
> then that is what people should be doing, no?
>
> Is there not some objective set of criteria for determining
> when it is, and when it is not, valid to run a small-volume correction?
>
> Many thanks,
>
> Raj
|