Hi Peter,
This is rather complicated because it depends on how the job is
submitted (lcg-RB and glite-WMS jobs have differnet wrapper scripts, and
so behave quite differently).
For lcg-RB jobs it's quiite simple, if you set $EDG_WL_SCRATCH to
$TMPDIR then the job will start there. The glite-WMS jobs are more
complicated. You need to set up a local customisation directory, pointed
to by GLITE_LOCAL_CUSTOMIZATION_DIR, and a script in it called cp_1.sh
which is run before the user payload and does the cd $TMPDIR.
Are you using the torque_mom $tmpdir comfig setting? This creates a
tmpdir per job in the temp directory and cleans it up at the end of the
job, and is better at it than the grid wrappers.
for those using cfengine my config looks like:
worker::
{ /opt/glite/etc/profile.d/grid-env.sh
BeginGroupIfNoLineMatching 'gridenv_set
"EDG_WL_SCRATCH" "$TMPDIR"'
ResetSearch "1"
LocateLineMatching 'gridenv_set
"EDG_WL_SCRATCH".*'
ReplaceLineWith 'gridenv_set "EDG_WL_SCRATCH"
"$TMPDIR"'
EndGroup
BeginGroupIfNoLineMatching 'gridenv_set
"GLITE_LOCAL_CUSTOMIZATION_DIR".*'
ResetSearch "1"
LocateLineMatching 'fi'
IncrementPointer "-1"
InsertLine 'gridenv_set
"GLITE_LOCAL_CUSTOMIZATION_DIR" "/opt/glite/ppd/"'
EndGroup
}
{ /opt/glite/ppd/cp_1.sh
AutoCreate
AppendIfNoSuchLine "cd $TMPDIR"
}
{ /var/spool/pbs/mom_priv/config
AutoCreate
PrependIfNoSuchLine "$clienthost heplnx201.pp.rl.ac.uk"
AppendIfNoSuchLine "$restricted heplnx201.pp.rl.ac.uk"
AppendIfNoSuchLine "$logevent 255"
AppendIfNoSuchLine "$tmpdir /scratch"
DefineClasses startmom
}
Yours,
Chris.
> -----Original Message-----
> From: Testbed Support for GridPP member institutes
> [mailto:[log in to unmask]] On Behalf Of Peter Love
> Sent: 23 June 2008 12:38
> To: [log in to unmask]
> Subject: WN pwd and torque TMPDIR
>
> Hey, does anybody have details on the expect behaviour of glite in
> regard to job working directories?
>
> Using torque, we setup TMPDIR to be a large scratch space and I
> understood torque will set this as cwd for each job. However, if I run
> this on various EGEE sites it returns a /home location:
> $ globus-job-run
> fal-pygrid-18.lancs.ac.uk:2119/jobmanager-lcgpbs -queue atlas /bin/pwd
>
> Most grid jobs are running in $TMPDIR but HONE jobs are running in
> /home, and fail due to disk space issues (we expect them to run in
> TMPDIR) https://gus.fzk.de/ws/ticket_info.php?ticket=37498
>
> Do job scripts need an explicite 'cd $TMPDIR' ?
>
> Peter
>
|