All this assumes that "wall time" means the human perception of the
passage of /time/ from the start to the completion of the job,
regardless of all other considerations such as the number of cpus, load
etc.) If that's not the case, then "all bets are off", so to speak.
Have I got that right?
Steve
On 11/18/2014 11:19 AM, Stephen Jones wrote:
> Hi Alessandra,
>
> > Have all CREAM-CE sites enabled the multicore support (see below)?
>
> No changes have been made to our CREAM CEs (e.g. to support
> multi-core.) They run single core jobs on a TORQUE/Maui cluster. Our
> ARC/CE is setup to run both single core and multi-core on a Condor
> cluster. The accounting is published via ARC Jura. At our site, the
> wall and cpu timings of jobs on various node types is scaled to a
> standard normalised value (which is 10HS06 or 2500 SI00) in accordance
> with this procedure:
> https://www.gridpp.ac.uk/wiki/Publishing_tutorial#Scaling_factors .
>
> > (i.e. CPUtime> walltime)
>
> "Blue" has occurred since we started to run multi-core jobs on our ARC
> CE. As Daniel says, it seems logical to suppose that cpu time for a
> job that runs on eight cpus for X minutes (elapsed time from a clock
> on the wall) wouldbe eight times as much as that for a job that runs
> for X minutes on only one cpu. So it is easy to see why cpu time
> easily exceeds walltime on multicore jobs, and the graph turns blue.
>
> If doubt exists about the integrity of the ARC/Condor/Jura/APEL data
> flow (as Andrew suggests), I'd ask these questions. Do developer tests
> show that ARC/Condor/Jura/APEL data flow is correct/coherent for both
> single and multicore jobs? Is it true and correct that (for muti-core
> jobs) the cpu time is expected to be larger than the elapsed wall time?
>
> Cheers,
>
> Steve
>
>
>
> On 11/14/2014 01:23 PM, Alessandra Forti wrote:
>> Dear UK sites,
>>
>> could you also go through this exercise please and check the
>> accounting numbers are correct for your site? I particular I see few
>> blue boxes in the efficiency [1] (i.e. CPUtime> walltime)
>>
>> Have all CREAM-CE sites enabled the multicore support (see below)?
>>
>> The CESGA portal maybe the problem as PIC is stating but I'd like to
>> iron out any misconfiguration before.
>>
>> thanks
>>
>> cheers
>> alessandra
>>
>> [1] http://tinyurl.com/kwlwang
>>
>>
>>
>> -------- Original Message --------
>> Subject: Multicore accounting
>> Date: Fri, 14 Nov 2014 12:06:58 +0000
>> From: Alessandra Forti <[log in to unmask]>
>> To: WLCG Multicore Deployment TF <[log in to unmask]>,
>> Rob Quick <[log in to unmask]>
>>
>>
>>
>> Dear all,
>>
>> this is another thread we need to check. We have receipes to enable
>> accounting for CREAM CEs and ARC-CEs should be working out of the box,
>> but I'd like sites to check their status. There seem to be some
>> discrepancies at some sites. We'd like to understand where they are
>> coming from i.e. have the sites published all the data? Have they
>> enabled the multicore counting in CREAM-CE [1]? Is it a portal problem?
>>
>> Next week we have been asked to report at the WLCG MB about this so any
>> info about discrepancies we need to investigate (or also if it works
>> perfectly) you can send us is much appreciated.
>>
>> thanks
>>
>> cheers
>> alessandra
>>
>> [1]
>> https://twiki.cern.ch/twiki/bin/view/LCG/MulticoreTFReports#Report_18_09_2014
>>
>>
>> --
>> Respect is a rational process.
>>
>>
>>
>
>
--
Steve Jones [log in to unmask]
System Administrator office: 220
High Energy Physics Division tel (int): 42334
Oliver Lodge Laboratory tel (ext): +44 (0)151 794 2334
University of Liverpool http://www.liv.ac.uk/physics/hep/
|