Dear all,
Turns out the problem is related to two singularity settings (in
singularity.conf):
mount home = yes
mount tmp = yes
if you try forcing all condor jobs to run within singularity.
The first once mounts $HOME for the condor user (!) which indeed is not
the submit-user.
The second interferes with condors MOUNT_UNDER_SCRATCH option if you are
using that for an isolated tmp.
All of this is a non-issue for jobs that call singularity themselves, it
is just a problem with
# Forces _all_ jobs to run inside singularity.
SINGULARITY_JOB = true
Cheers,
Luke
On 25/04/18 15:01, Luke Kreczko wrote:
> Hi Alessandra,
>
> Thank you very much for the reply.
>
>
> On 25/04/18 14:38, Alessandra Forti wrote:
>> Hi Lukas,
>>
>> I don't think anyone else is using unprivileged singularity in the
>> UK, and I don't think anyone is using it as part of condor either.
>> The latter at least is incompatible with running singularity as part
>> of the pilot infrastructure.
>
> In that case I will stick to the privileged setup.
>
> Cheers,
>
> Luke
>
>>
>> cheers
>> alessandra
>>
>> On 25/04/2018 10:33, Luke Kreczko wrote:
>>> Dear all,
>>>
>>> If nobody has seen this problem before, would it be possible for
>>> sites running HTCondor + singularity to send their relevant HTCondor
>>> config bits (e.g. [1])?
>>>
>>> Cheers,
>>>
>>> Luke
>>>
>>>
>>> [1]
>>> condor_config_val -dump | grep -i singularity
>>> SINGULARITY =
>>> /cvmfs/oasis.opensciencegrid.org/mis/singularity/el7-x86_64/bin/singularity
>>> SINGULARITY_BIND_EXPR =
>>> "/hdfs,/software,/users,/usersc,/condor/condor,/usr/libexec/condor"
>>> SINGULARITY_IMAGE_EXPR = (TARGET.DESIRED_OS is "CentOS6") ?
>>> "/cvmfs/cernvm-prod.cern.ch/cvm3" :
>>> "/cvmfs/cms.cern.ch/rootfs/x86_64/centos7/latest"
>>> SINGULARITY_JOB = true
>>> SINGULARITY_TARGET_DIR = /srv
>>> STARTER_JOB_ENVIRONMENT = "SINGULARITY_HOME=/srv"
>>>
>>>
>>> On 20/03/18 11:10, Luke Kreczko wrote:
>>>> Dear all,
>>>>
>>>> I am currently trying to configure Singularity (2.4.2) and HTCondor
>>>> (8.6.10) on a CentOS7 WN (7.4, kernel 3.10.0-693.21.1).
>>>>
>>>> I've followed the instructions in
>>>>
>>>> http://opensciencegrid.github.io/docs/worker-node/install-singularity/#unprivileged-singularity
>>>>
>>>>
>>>> https://research.cs.wisc.edu/htcondor/manual/latest/3_17Singularity_Support.html
>>>>
>>>>
>>>> and rebooted the machine.
>>>>
>>>> For testing I run everything in the
>>>>
>>>> /cvmfs/cernvm-prod.cern.ch/cvm3
>>>>
>>>> image.
>>>>
>>>> I can execute all examples on the node as both root and local user,
>>>> HTCondor jobs work under native CentOS7, but enabling singularity
>>>> leads to
>>>>
>>>> ERRORĀ : Home directory is not owned by calling user: /srv
>>>> ABORTĀ : Retval = 255
>>>>
>>>>
>>>> I assume that singularity is started as HTCondor but /srv is owned
>>>> by the submitting user.
>>>>
>>>> Has anyone seen this before and does know the solution?
>>>>
>>>>
>>>> Cheers,
>>>>
>>>> Luke
>>>>
>>>>
>>>> [1]
>>>>
>>>> SINGULARITY =
>>>> /cvmfs/oasis.opensciencegrid.org/mis/singularity/el7-x86_64/bin/singularity
>>>> SINGULARITY_BIND_EXPR =
>>>> "/hdfs,/software,/users,/usersc,/condor/condor,/usr/libexec/condor"
>>>> SINGULARITY_IMAGE_EXPR = "/cvmfs/cernvm-prod.cern.ch/cvm3"
>>>> SINGULARITY_JOB = true
>>>> SINGULARITY_TARGET_DIR = /srv
>>>> STARTER_JOB_ENVIRONMENT = "SINGULARITY_HOME=/srv"
>>
To unsubscribe from the TB-SUPPORT list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=TB-SUPPORT&A=1
|