Dear Anderson,
To make it clear to me, I wonder where could be the difference between FreeSurfer and PALM implementation in my particular test case?
My test case uses orthogonal design matrix (i.e. no problems with partitioning) and both in FreeSurfer and in PALM the permutation of rows of design matrix is used for building the null distribution. The only difference I would expect is the difference in randomization of rows of the design matrix, i.e. difference in particular permutations used in building the null distribution. I used 50000 permutations in both cases, therefore this effect should be small (and indeed there is very close correspondence between p-values as I stated in previous mail).
Is here any other difference I am not aware of?
I was also wondering: is it possible to save built null distribution of cluster extent to a file so that I can compare the shape of null distributions generated by FreeSurfer and PALM?
Also, I tried to re-process my analysis of gyrification index with non-orthogonal design matrix (which was properly analyzable in FreeSurfer only by Monte Carlo null-distribution). The p-values from PALM are in all cases much higher than in FreeSurfer where Monte Carlo simulation of null-distribution was used. (Cluster forming threshold was p=0.05, two-tail hypothesis).
For example, in case, where FreeSurfer showed p=0.0002, PALM shows p=0.18. Could you expect such result?
I suspect that the Monte Carlo simulation of null distribution does not control for familly-wise error rate sufficiently. I tried to compute familly-wise error rate by permuting rows of design matrix and repeatably applying cluster inference by pre-cached Monte Carlo simulation. The results showed me that clusters with p=0.05 are present in much more than 5% cases (for cluster forming threshold p=0.05 the apparent FWE rate was up to 30%, for higher cluster forming threshold the apparent FWE was closer to expected value). On the other hand, isn't permutation approach in PALM overly conservative? Could you please comment on?
Regards,
Antonin Skoch
|