Hi Luke,
We used MOUNT_UNDER_SCRATCH for 3-4 years without problems. It was essential to use the lcmaps plugin in order for glexec (i.e. CMS) jobs to work.
We only stopped using MOUNT_UNDER_SCRATCH when moving to the Docker universe, as Docker obviously provides unique /tmp and /var/tmp per job.
Regards,
Andrew.
________________________________
From: Testbed Support for GridPP member institutes [[log in to unmask]] on behalf of Luke Kreczko [[log in to unmask]]
Sent: Tuesday, December 05, 2017 9:16 AM
To: [log in to unmask]
Subject: HTCondor's MOUNT_UNDER_SCRATCH and pilot jobs
Dear all,
We've recently encountered a problem where CMS jobs (HammerCloud) where failing due to a segmentation fault.
This has been traced down to an attempt of the CMS SW to write to /tmp and failing to do so.
We used MOUNT_UNDER_SCRATCH which results /tmp and /var/tmp to be mounted in the condor job directory and it works for normal jobs.
But for pilot jobs these folders are owned and write-only to the pilot account and not the payload user.
When searching for possible solutions I've encountered an outdated lcmaps-plugins-mount-under-scratch module:
https://github.com/bbockelm/lcmaps-plugins-mount-under-scratch-historical
which solved the issue by creating additional folders which are then owned by the payload user.
What are people using nowadays? Is nobody using MOUNT_UNDER_SCRATCH or is it only used in conjunction with Singularity?
Cheers,
Luke
|