Hi Maarten,
Maarten Litmaath, CERN wrote:
> On Wed, 21 Mar 2007, Ilja Livenson wrote:
>>Sorry, but what does scalability mean in this case? Less memory?
>
> Less CPU consumption when the CE handles many hundreds of jobs in parallel.
Sorry if I'm mistaken, but isn't the biggest improvement in this
area due to the use of the "grid_manager" process replacing on-the-fly
the traditional GT2-provided job manager, for jobs submitted through the
RB?
Whether you use lcgpbs or pbs is largely irrelevant in this scenario,
as either will get slaughtered by the grid_manager for RB-submitted jobs.
We have and are running happily with the pbs job manager on our CE at
NIKHEF, with queued+running job counts up to 2000 jobs simultaneously
at any single CE, and have never seen any problems with the pbs job
manager.
It's only when people start submitting directly to the CE that the
system gets swamped -- and I'm not sure the use of the lcgpbs job manager
will help a lot in that case, since it still requires one-process-per-
job-queued-or-running to keep the job state, isn't it?
Cheers,
DavidG.
>
>
>>Maarten Litmaath wrote:
>>
>>>Stephen Childs wrote:
>>>
>>>
>>>>We've just changed over to the pbs job manager from the lcgpbs
>>>>jobmanager in order to fit in with the RB restrictions on MPI jobs
>>>>and I've noticed a massive speedup.
>>>>
>>>>A simple globus-job-run completes in < 25 seconds (compared to around
>>>>2 minutes for lcgpbs) and a job submitted through the RB completes in
>>>>just over 2 minutes (compared to around 5-6 minutes for lcgpbs).
>>>>
>>>>I thought this change wouldn't make a huge difference as I thought
>>>>most of the delay was in the RB. Has anyone else noticed this or was
>>>>there something horribly wrong with our lcgpbs jobmanager that was
>>>>causing the slowdown.
>>>
>>>The "lcg" job managers have been coded for scalability instead of speed.
>>>Normally those few minutes of delay are irrelevant.
>>
--
David Groep
** National Institute for Nuclear and High Energy Physics, PDP/Grid group **
** Room: H1.56 Phone: +31 20 5922179, PObox 41882, NL-1009DB Amsterdam NL **
|