Dear Chunliang,
Can you provide additional information what the reviewer proposed, as it might clarify what he doesn't like exactly? I mean, it's in the very nature that post-hoc tests are dependent on previous results. In a simple case, some within-subject 2x2 ANOVA on reaction times resulting in a sig. interaction will likely mean you conduct a series of pairwise comparisons on RTs from the four conditions. In general you will only run post-hoc tests if there were sig. results in the ANOVA in the first place. There are problems with post-hoc tests, yes, but they are really very common, and as long as you explicitely state they are post-hoc tests (and account for multiple testing where necessary) it should be okay.
When it comes to massive univariate statistics it's somewhat more complicated and there's no real consensus. However, as you relied on cluster correction it is reasonable to look at cluster-averaged beta estimates (in contrast to e.g. those from the peak voxel). It is problematic insofar as averaging across voxels should also increase SNR, thus in contrast to the example with reaction times, the post-hoc tests are not conducted on the very same data. But there's no simple solution. E.g. you could run post-hoc tests for every single voxel within the cluster, but this would conflict with the previous cluster correction. So cluster-averaged beta estimates (or first eigenvariate) seems to be a good option.
> and we want to confirm that the interaction was driven by the differences between them
This is problematic IMO, as it sounds as if you just focus on some of the comparisons, meaning you seem to have a priori hypotheses, which conflicts with the idea of post-hoc tests (as they are a posteriori). Maybe this is what the reviewer doesn't like.
Hope this helps a little
Helmut
|