Hello
I've run MELODIC on a few datasets now to test reproducibility between
multiple execs on my machine and multiple execs on another platform, and I
find that there is serious variability, from one result to the next obtained
within 30 seconds of each other.
Running MELODIC from the GUI, I create a configuration and disable slice
timing correction, volume registration, blurring, and spat norm (I have done
these externally), and retain the highpass of 100s. I hit Go. Browser pops
up and report is generated live as usual.
Then, without closing the MELODIC_GUI, I only change the output directory
to something like 'trial2', and hit 'Go' again around 30 seconds after.
I view the output simultaneously.
Prestats are identical, and PCA eigenvalue curves are the same.
Comparing between the two execs, usually the first 10 or so components are
comparable but in different orders.
The components that are similar have variances percentages that can easily
be 10% different from one another. While they are roughly similar, the
timecourses and power spectra have differing subtle features, and spatial
localization maps are different too.
A few 'neural' components are within the first 10, most are motion/artifact.
As we go to the lower components we find more 'neural' components, but where
some components are clear and pretty in one decomposition, they may be
completely absent or heavily intermingled with high freq noise in another.
I tried this on my Mac, then I tried this on a linux box. Same effect.
What's going on?
-Prantik
|