Unfortunately I've been a bit under the weather and not really followed up
on this yet.
On Wed, 18 Feb 2009, Burke, S (Stephen) wrote:
> Testbed Support for GridPP member institutes
> > [mailto:[log in to unmask]] On Behalf Of Henry Nebrensky said:
> > The ERTs being published are correct, very different (short
> > is currently zero!), and none of them is 2196.
>
> Bear in mind that you need to look at the ERT for the mice VOView, not
> the CE as a whole.
Well, surely I need to look at the the ones that reflect what the actual
job will encounter - e.g. that the short queue (ERT=0) has an idle CPU
sitting and waiting - rather than some non-representative averaged value.
There isn't any way to specify which view to use to the matching process,
is there?
> If I look now the three queues don't show 2196, but
> they are all identical (7520, and indeed they were 7308 a few minutes
> ago). So it is updating, but may or may not be doing what you expect ...
> as far as I remember the algorithm is based on the recent history of how
> long actual jobs take to run, but it looks like it must be using the
> same history for all the queues. If you really think it's wrong Jeff
> Templon is the author and may be able to help more.
It looks like it is indeed intended, as
/opt/lcg/libexec/lcg-info-dynamic-scheduler includes a comment that
# this is tricky: reporting job counts in queues cares about queues,
# but scheduling doesn't as far as I know.
This approach seems undermine the ability of the WMS to do the
matchmaking: queues that should rank highly are given a poor ranking
because of the presence at the cluster of jobs in other classes. (The
converse also appears to occur - if the VO has no jobs queued, then the
ERT will be published as an attractive zero even if the queues are already
filled with other VOs jobs).
This effect won't show up on sites that have single per-VO queues, but it
seems that an increasing number are deploying shared queues with different
time limits.
Has anyone else noticed issues that might be connected with this?
Thanks
Henry
--
Dr. Henry Nebrensky [log in to unmask]
http://people.brunel.ac.uk/~eesrjjn
"The opossum is a very sophisticated animal.
It doesn't even get up until 5 or 6 p.m."
|