cheers, chris. I'll try this here... and update the examples. You're
right - the expression we resorted to is a horror!
Ste
On 2019-10-23 10:40, Chris Brew - UKRI STFC wrote:
> Dear All,
>
> The example GridPP setup has a pretty horrible (and hard to maintain)
> chain of nested ifThenElse statements to assign a job to an accounting
> group. Since, the example config was written, Schedd Job transforms
> and usermaps have been added as a feature and after discussions at the
> recent Condor workshop in Italy I decided to try to get rid of the
> horror.
>
> Our setup was even worse because we run local jobs as well and have an
> extra tier in the accounting groups (the Partition) to split between
> local and grid jobs but you should be able to get the drift. I've also
> slightly changed to format, putting the multicore/singlecore into its
> own level*.
>
> Yours,
> Chris.
>
> * Next step is to try to get rid of this altogether, suggestion is to
> change the START expression on the workers to only accept multicore
> jobs for several cycles after a multicore slot becomes free. That
> means you don't need to negotiate multicore jobs first, so they don't
> need their own accounting groups.
>
> ```
> $ cat config.d/80grid_accounting_groups
> SCHEDD_CLASSAD_USER_MAP_NAMES = $(SCHEDD_CLASSAD_USER_MAP_NAMES) VOMap
> VORoleMap
> CLASSAD_USER_MAPFILE_VOMap = /etc/condor/maps/vo_map
> CLASSAD_USER_MAPFILE_VORoleMap = /etc/condor/maps/vo_role_map
> # Assign groups automatically
> JOB_TRANSFORM_NAMES = $(JOB_TRANSFORM_NAMES) AssignGroupGrid
>
> JOB_TRANSFORM_AssignGroupGrid @=end
> NAME AssignGroupGrid
> COPY Owner TestAcctGroupUser
> COPY VOName x509UserProxyVOName
> SET TestPartition "group_GRID"
> EVALSET TestAcctGroup usermap("VOMap",x509UserProxyVOName)
> EVALSET TestAcctRole usermap("VORoleMap",TestAcctGroupUser)
> EVALSET JobType ifThenElse( RequestCpus > 1, "multicore",
> "singlecore");
> EVALSET TestAccountingGroup strcat(TestPartition, ".",
> TestAcctGroup, ".", x509UserProxyVOName, "_", TestAcctRole, ".",
> JobType, ".", TestAcctGroupUser);
> @end
>
> # Prevent Cheating
> IMMUTABLE_JOB_ATTRS = $(IMMUTABLE_JOB_ATTRS) TestAccountingGroup
> TestPartition TestAcctGroup TestAcctGroupUser
>
> $cat maps/vo_map
> * atlas ATLAS
> * cms CMS
> * lhcb LHCB
> * dune DUNE
> * mice MICE
> * lz LZ
> * dteam DTEAM_OPS
> * ops DTEAM_OPS
> * /.*/ OTHER
>
> $cat maps/vo_role_map
> * /prd.*/ prod
> * /plt.*/ pilot
> * /pdlhcb.*/ prod
> * /pllhcb.*/ pilot
> * /.*/ user
>
> $ condor_q -all -af AccountingGroup TestAccountingGroup | tail -10
> group_GRID.LHCB.lhcb_pilot.pllhcb20
> group_GRID.LHCB.lhcb_pilot.singlecore.pllhcb20
> group_GRID.LHCB.lhcb_pilot.pllhcb20
> group_GRID.LHCB.lhcb_pilot.singlecore.pllhcb20
> group_GRID.LHCB.lhcb_pilot.pllhcb20
> group_GRID.LHCB.lhcb_pilot.singlecore.pllhcb20
> group_GRID.LHCB.lhcb_pilot.pllhcb20
> group_GRID.LHCB.lhcb_pilot.singlecore.pllhcb20
> group_GRID.LHCB.lhcb_pilot.pllhcb20
> group_GRID.LHCB.lhcb_pilot.singlecore.pllhcb20
> group_GRID.CMS.cms_pilot_multicore.pltcms07
> group_GRID.CMS.cms_pilot.multicore.pltcms07
> group_GRID.CMS.cms_pilot_multicore.pltcms07
> group_GRID.CMS.cms_pilot.multicore.pltcms07
> group_GRID.CMS.cms_pilot_multicore.pltcms07
> group_GRID.CMS.cms_pilot.multicore.pltcms07
> group_GRID.CMS.cms_pilot_multicore.pltcms07
> group_GRID.CMS.cms_pilot.multicore.pltcms07
> group_GRID.LHCB.lhcb_prod.pdlhcb19
> group_GRID.LHCB.lhcb_prod.singlecore.pdlhcb19
> ```
>
> --
> Dr Chris Brew
> Scientific Computing Manager
> Particle Physics Department
> UKRI - STFC - Rutherford Appleton Laboratory
> Harwell Oxford,
> Didcot
> OX11 0QX
> +44 1235 446326
>
>
>
> ########################################################################
>
> To unsubscribe from the TB-SUPPORT list, click the following link:
> https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=TB-SUPPORT&A=1
########################################################################
To unsubscribe from the TB-SUPPORT list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=TB-SUPPORT&A=1
|