 |
 |
Thank you Pierre for these useful suggestions: with all of these inputs I have the feeling you guys almost ran my analysis ;-) Bruno On 28/01/2014 17:57, Pierre Bellec wrote: > Hi Bruno, > > Looks like Donald got you covered. For future reference, if you can't > avoid using files for a simulation and want to be as efficient as > possible, the simplest hack is to store those files on a virtual > partition in memory instead of a hard drive. If you are using a > high-performance computing infrastructure, chances are this is available > to you or can be arranged with the staff. You would have to make sure > that the intermediate outputs fit in the virtual partition, but if we > are talking about one parameter map at a time this will be no problem. > > Also, if you are writing all files in the local drive of your compute > node (the tmp), you won't be clogging the grid (because you don't put > any load on the shared file system). > > Finally, the computation time you will save by not using the disk may > not actually be that large. You may want to start doing a bit of > profiling (with the profile command) on a small simulation to check how > big the overhead of using files actually is, rather than assume it will > be large. Matlab is a weird beast, and computational bottlenecks are not > necessarily where you would expect them to be. > > I hope this helps, > > Pierre Bellec, PhD > Researcher / Chercheur, Research Centre of the Montreal Geriatric Institute > Professeur adjoint sous octroi, Department of Computer Science and > Operations Research > University of Montreal, Québec, Canada > 1 514 340 3540 #3367 > http://simexp-lab.org/brainwiki/doku.php?id=pierrebellec > > > 2014-01-28 Bruno L. Giordano <[log in to unmask] <mailto:[log in to unmask]>> > > Hello, > > I am planning to run some permutation analysis by permuting > condition labels in the first-level model. > > I wonder whether someone managed to hack the first-level model > routines so that they take in input and give in output variables > stored in the workspace rather than stored on the disk (I would > actually need only the parameter estimates, and not the stats, which > should make things much easier). I suspect that in the case of a > permutation analysis disk access is going to be a considerable > limiting factor for the speed of the computations, and I wouldn't > like to overload the parallel grid with unnecessary overheads. > > Thank you for any tip. > > Bruno > > ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~__~~ > Bruno L. Giordano, PhD > Institute of Neuroscience and Psychology > 58 Hillhead Street, University of Glasgow > Glasgow, G12 8QB, Scotland > T +44 (0) 141 330 5484 <tel:%2B44%20%280%29%20141%20330%205484> > Www: http://www.brunolgiordano.net > > -- ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Bruno L. Giordano, PhD Institute of Neuroscience and Psychology 58 Hillhead Street, University of Glasgow Glasgow, G12 8QB, Scotland T +44 (0) 141 330 5484 Www: http://www.brunolgiordano.net