Dear Jesper,
Thanks, I will try. If I have a problem I will come back.
Best,
Jun
> 差出人: Jesper Andersson <[log in to unmask]>
> 件名: Re: EDDY parallelization
> 日付: 2015年9月4日 18:18:01 JST
>
>
> Dear Jun,
>
>>
>> Sorry for interrupting. Eddy on my MacPro (24 core) doesn't get parallelized either.
>
> It doesn’t sound like the same “problem" that Derek described. If I understood him correctly his jobs got parallelised on a given node, but not across multiple nodes.
>
>> I tried setting OMP_NUM_THREADS to 24, but it did not work. What is the system requisite for Eddy to be parallelized?
>
> For eddy to run in parallel it needs to be built with the openmp flag. Unfortunately Apple ships a very old version of gcc (4.2) and full (not so buggy) openmp support only came to gcc with version 4.4. If you want to build a parallel version of eddy on MacOS you need to upgrade your gcc to at least 4.4 (I am currently on 4.8). It is a little fiddly and I could only do it thanks to the support of the IT boys.
>
> If you do build an openmp version, you don’t want to give all your cores to eddy. Back when I was still running the openmp version of eddy on my laptop I always saved a couple of cores for interactive work.
>
> Good luck Jesper
>
>>
>> Thanks in advance,
>>
>> Jun
>>
>> ************************************************
>> Jun Miyata, M.D., Ph.D.
>> Senior Lecturer
>> Department of Psychiatry
>> Graduate School of Medicine
>> Kyoto University
>> 54 Shogoin-Kawaharacho, Sakyo-ku
>> Kyoto 606-8507, Japan
>> Tel: +81-75-751-3386
>> Fax: +81-75-751-3246
>> E-mail: [log in to unmask]
>> ************************************************
>>
>>> 差出人: Derek Pisner <[log in to unmask]>
>>> 件名: Re: EDDY parallelization
>>> 日付: 2015年9月3日 4:14:10 JST
>>>
>>>
>>> Hi Jesper,
>>>
>>> Thanks for the clarification- makes total sense now.
>>>
>>> Derek
>>> ________________________________________
>>>> From: FSL - FMRIB's Software Library [[log in to unmask]] on behalf of Jesper Andersson [[log in to unmask]]
>>>> Sent: Tuesday, September 01, 2015 2:59 AM
>>>> To: [log in to unmask]
>>>> Subject: Re: [FSL] EDDY parallelization
>>>>
>>>> Hi Derek,
>>>>
>>>> these are different types of parallelisations. bedpostx, randomise_parallel etc are all solving problems that are essentially a large number of 1D problems. Hence it is relatively straightforward to divide it up into slices (or whichever chunk one prefers) and then send these to different nodes.
>>>>
>>>> Eddy on the other hand is a 4D problem, and hence cannot be divided up like that. Therefore it uses OpenMP instead, which means that all threads have to reside on the same node and have access to the same heap/RAM. The work will be divided up over all the cores on the node it is submitted to, hence speeding things up. BUT, if you submit several eddy jobs on the same node they will all compete for the cores on that node and it will be no faster than if you had submitted the eddy jobs sequentially.
>>>>
>>>> If you want to limit the number of cores that an OpenMP job can bogart you can set the environment variable OMP_NUM_THREADS to a suitable number. So for example if you set OMP_NUM_THREADS to 4, you could have six eddy jobs happily using 4 cores each on your 24 core machine. But unless you have sufficient RAM to run all of those eddy processes at the same time it might still be a better idea to run the sequentially, letting each job use all 24 cores.
>>>>
>>>> Jesper
>>>>
>>>> On 31 Aug 2015, at 17:25, Derek Pisner <[log in to unmask]> wrote:
>>>>
>>>>> The Eddy documentation says that eddy has been configured to utilize openMP, but when I try to submit eddy as a parallel job using SGE, the job always sends to one queue on one node and overloads all of my cpu's (I have a 24 core machine). Is there currently a beta eddy_parallel command floating around somewhere? Or does fsl_sub need to be modified in some further way to allow Eddy to round_robin across multiple nodes like bedpostx, randomize_parallel, and other fsl parallelization functions do?
>>>>>
>>>>> Thanks,
>>>>> Derek
|