Bonjour Jean-Michel,
> Last week, I switched the jobmanager from lcgpbs to pbs on en LCG-CE.
>
> It seems that with this new pbs jobmanager, the EDG_WL_SCRATCH
> that I use to direct the jobs to use local storage /dlocal on the
> worker nodes is no longer taken into account. The working directories
> are created in the home directory of the pool accounts and not under
> /dlocal.
Neither lcgpbs nor the pbs job manager care about EDG_WL_SCRATCH.
That variable is taken into account only by the lcg-RB job wrapper:
----------------------------------------------------------------------
if [ ! -z "$EDG_WL_SCRATCH" ]; then
cd $EDG_WL_SCRATCH
cleanupDir
fi
newdir=WMS_${myhost}_0$$_https_3a_2f_2f..... # etc.
initdir=`pwd`
mkdir ${newdir}
cd ${newdir}
----------------------------------------------------------------------
The gLite WMS job wrapper has this customization code instead:
----------------------------------------------------------------------
# customization point #1
if [ -n "${GLITE_LOCAL_CUSTOMIZATION_DIR}" ]; then
if [ -f "${GLITE_LOCAL_CUSTOMIZATION_DIR}/cp_1.sh" ]; then
. "${GLITE_LOCAL_CUSTOMIZATION_DIR}/cp_1.sh"
fi
fi
----------------------------------------------------------------------
The CREAM job wrapper essentially has the same code.
So, if your WN defines GLITE_LOCAL_CUSTOMIZATION_DIR, it will look for
a script ${GLITE_LOCAL_CUSTOMIZATION_DIR}/cp_1.sh in which you can let
the job do something like this:
----------------------------------------------------------------------
d=`mktemp -d /dlocal/job-XXXXXXXXXX` || exit
export HOME=$d
cd $d || exit
----------------------------------------------------------------------
> This causes problems with disk space because the home directories
> are NFS mounted and the alice working space is big.
>
> => Is there anyone using successfully EDG_WL_SCRATCH with pbs
> jobmanager ? Is this the right way to use local storage at
> worker nodes ?
Arguably the lcgpbs and pbs job managers ought to support the notion
of running the job in a $TMPDIR subdirectory whenever possible.
You could open a bug about that, but the gLite customization trick
may well prove sufficient.
|