Hi Ewan
Short answer: It’s a thing that is still in development. I do not believe it will be useful in the scenario you describe for a long time.
Long answer:
The Event Service has been in development for a while and is a US project., the latest talks can be found:
https://indico.cern.ch/event/342709/contribution/1/material/slides/0.pdf
and
https://indico.cern.ch/event/276499/session/10/contribution/0/material/slides/0.pdf
There is a quite detailed wiki page here:
https://twiki.cern.ch/twiki/bin/view/PanDA/EventServer
I am not sure how much of that is public, but from what I understand the idea of the event service is to use it with opportunistic HPC.
On a related note, the Ceph instance at BNL is part of this Event Service scheme. I am not entirely sure what they are doing and it would probably be easier finding out what the X-37B is up to!
On yet another related note, I believe the recent massive over usage of Oxford Scratchdisk was related to the event picking service (which seems to be very similar). If somebody wants to run on a small number of events spread over lots of files/datasets, Panda can make a temporary dataset with the files that are needed and then copy them all to one site when the job is then run. This is fine when the total is small but it has gone spectacularly wrong on a few occasions, so if people notice their Scratchdisk being suddenly used up this should be the first line of investigation.
Alastair
On 16 Oct 2014, at 17:37, Ewan MacMahon <[log in to unmask]> wrote:
> Hi all,
>
> In discussions of how to handle the unpredictable bits of spare time available when (for example) draining a worker node to accommodate a multicore job, mentions have been made of an ATLAS 'Event Server' that would allow a machine to run jobs in very small (essentially one event at a time) chunks, which would allow it to accomplish useful work in whatever time was available and then be terminated at short/no notice without losing too much.
>
> I've had a quick Google and turned up very little beyond a link to a non-existent twiki page, so - is the event server a thing? Used to be a thing? Is expected to be a thing? Just an idea? Something else?
>
> Ewan
|