Hi John,
Thanks for looking, our overall cluster load has gone form it's usual 5K-ish to 15K :)
Now that's reported more than actual as the cgroup share is causing them to not affect other running processes so I'm not too worried, but if we hadn't had cgroups I think I'd be kill jobs right about now.
Thanks,
Gareth
-----Original Message-----
From: Testbed Support for GridPP member institutes [mailto:[log in to unmask]] On Behalf Of John Hill
Sent: 16 November 2017 12:47
To: [log in to unmask]
Subject: Re: LHCb multiothreaded user jobs
I see a handful of these at Cambridge (few enough that I wouldn't have noticed if I hadn't actively looked).
John
On 16/11/2017 12:37, Gareth Roy wrote:
> Hi All,
>
> We're currently seeing multithreaded payloads from LHCb that are causing high loads on some of our batch farm (although contained somewhat by cgroups). It appears a user has compiled some code enabling openmp (see runjob script below), so the job is attempting to start and run 32 threads per job.
>
> Is anyone else seeing these payloads? We're reasonably well protected via cgroups but it does mean that this user has 32 threads sharing a single cpu share so the jobs may take a while to complete (and potential waste cpu time).
>
> Thanks,
>
> Gareth
>
>
> ================================================
> # cat runjob.sh
> #!/bin/bash
> set -euo pipefail
> IFS=$'\n\t'
> set -x
>
> ls -la /cvmfs/
> source
> /cvmfs/sft.cern.ch/lcg/external/gcc/6.2.0/x86_64-slc6-gcc62-opt/setup.
> sh
> g++ -o energy_test -std=c++1z -O3 -Wall -I. energy_test.cpp -fopenmp
>
> ENERGY_TEST_SEED=$(od -N 6 -t uL -An /dev/urandom | tr -d " ") echo
> "Using seed ${ENERGY_TEST_SEED}"
> ./energy_test --max-events=500000 --max-permutation-events-1=$1
> --seed=$ENERGY_TEST_SEED --n-permutations=$2 sample1.txt sample2.txt
> --permutations-only
>
> [University of Glasgow: The Times Scottish University of the Year
> 2018]
>
|