Hello,
Thanks, this fixed the problem.
Best regards,
Stijn
Antun Balaz wrote:
> Hi Stijn,
>
> The attached files, if put in /etc/profile.d/ on CE and all WNs, will
> hopefully take care of this problem :)
>
> Best regards, Antun
>
> -----
> Antun Balaz
> Research Assistant
> E-mail: [log in to unmask]
> Web: http://scl.phy.bg.ac.yu/
>
> Phone: +381 11 3160260, Ext. 152
> Fax: +381 11 3162190
>
> Scientific Computing Laboratory
> Institute of Physics, Belgrade, Serbia
> -----
>
> ---------- Original Message -----------
> From: Stijn De Smet <[log in to unmask]>
> To: [log in to unmask]
> Sent: Mon, 23 Apr 2007 14:01:29 +0200
> Subject: Re: [LCG-ROLLOUT] MPI and EDG_WL_SCRATCH
>
>> Hello,
>>
>> I installed the scripts, but they don't seem to help(I submit the
>> jobs using the MPICH jobtype). I added some lines to the pbs.pm
>> script, and the directory looks
>> OK(/home/betest001//gram_scratch_VThCRN5Ajx). I saw that the pbs
>> jobtype used for this kind of jobs is 'single', so it starts a
>> single executable with a number of CPU's reserved for it. If this is
>> correct, then the Jobwrapper created at the resource broker is run
>> on the first node. This jobwrapper has some lines like this
>> included(almost at the start of the script after the function
>> definitions):
>>
>> if [ ! -z "$EDG_WL_SCRATCH" ]; then
>> cd $EDG_WL_SCRATCH
>> cleanupDir
>> fi
>>
>> newdir="https_3a_2f_2fgridrb.atlantis.ugent.be_3a9000_2frpFW4r8NnYsI4VjjIG41NQ"
>> mkdir -p ".mpi/"${newdir}
>>
>> So the directory my MPI job ends up in, is not ~/.... but /scratch/.mpi/...
>>
>> Regards,
>> Stijn
>>
>> Antun Balaz wrote:
>>> Hi Stijn,
>>>
>>> You need to replace /opt/globus/lib/perl/Globus/GRAM/JobManager/pbs.pm with
>>> the attached pbs.pm (of course, do the diff!). In order to avoid changing back
>>> to the old version of pbs.pm after CE reconfiguration, you may also want to
>>> replace /opt/globus/setup/globus/pbs.in with the attached. This pbs.pm will
>>> put all jobs that do not use more than 1 CPU to $EDG_WL_SCRATCH, so be sure to
>>> define it on all WNs! MPI jobs will stay in homes of pool accounts.
>>>
>>> I would also suggest that you replace
>>> /opt/globus/lib/perl/Globus/GRAM/JobManager.pm with the attached
>>> JobManager.pm, so that you avoid possible problems with proxy renewal (if
>>> /home is shared over NFS, sometimes it happens that the renewed proxy,
>>> although availble, is not seen on WNs).
>>>
>>> Hope this helps,
>>> Antun
>>>
>>>
>>> -----
>>> Antun Balaz
>>> Research Assistant
>>> E-mail: [log in to unmask]
>>> Web: http://scl.phy.bg.ac.yu/
>>>
>>> Phone: +381 11 3160260, Ext. 152
>>> Fax: +381 11 3162190
>>>
>>> Scientific Computing Laboratory
>>> Institute of Physics, Belgrade, Serbia
>>> -----
>>>
>>> ---------- Original Message -----------
>>> From: Stijn De Smet <[log in to unmask]>
>>> To: [log in to unmask]
>>> Sent: Fri, 20 Apr 2007 08:45:58 +0200
>>> Subject: [LCG-ROLLOUT] MPI and EDG_WL_SCRATCH
>>>
>>>> Hello,
>>>>
>>>> I recently configured MPI support on my nodes, but when I try to use
>>>> it, it always fails because even MPI jobs get started in the
>>>> EDG_WL_SCRATCH directory, which isn't shared, while my homedirs are.
>>>> My nodes are configured using YAIM, but for the moment, I don't use
>>>> the yaim mpi configuration. Is there an easy solution for disabling
>>>> the SCRATCH directory for MPI jobs, or do I just have to disable
>>>> scratch space completely?
>>>>
>>>> Regards,
>>>> Stijn
>>> ------- End of Original Message -------
> ------- End of Original Message -------
>
|