LHC Computer Grid - Rollout
> [mailto:[log in to unmask]] On Behalf Of Oxana Smirnova said:
> How's that? I need no infosys to query RLS; and RLS records
> have pretty explicit SFNs, don't they? globus-url-copy needs no infosys
either.

It's true that you can get the files out with low-level tools, but none of
the standard lcg-* and edg-rm things will work, and nor will jobs with input
files specified - and they will stop working for all users and all VOs. I
don't see any reason to do that, if you leave it in the info system a full
SE will still work perfectly well for file reads (at least assuming the
storage isn't shared with the root partition, which people have done from
time to time, in which case the machine will probably crash).

> Anyway, I am not really insisting on removing SEs from the infosystem;
> but do you know of any LCG tool or method that makes use of the
> published free space?

No, and I agree that it's something that should be in - I've been lobbying
for this kind of thing for a long time but there were always other things
considered more important ... however, it isn't that hard to put it in your
own code. Also you are supposed to be able to put a requirement on a minimum
free space on the close SE in the JDL, the so-called gang-matching, although
I'm not 100% certain that it works.

> And you actually mentioned yourself that what is
> published is the overall space, not a per-VO quota. I'm suggesting a
> least damaging solution (in my opinion), and I'm willing to discuss
> alternatives.

What the schema publishes is the free space per VO, but what that means
depends on the SE; some have a separate partition for each VO, some have all
VOs in one, and some have a mixture, e.g. a dedicated partition for "major"
VOs at that site and a single one for everyone else. [1] There's no explicit
way to know which it is, one of the many defects in the schema, but in
practice if two SAs have exactly the same free space you can assume that
they're shared - I believe that some of the monitoring tools make that
assumption. Chris Brew was suggesting putting VO quotas on a single
partition and hacking the info provider to return the remaining quota as the
free space.

[1] That may also be a significant point in itself, a site may provide a
large amount of dedicated space for cms if they are funded for cms, while
only giving atlas a bit of shared space as a courtesy.

> A full *system* partition is  fatal, but I am
> sure nobody has storage area and system area on the same partition.

I seem to remember that was the default EDG configuration at one time, but
we fixed it there and I hope no-one in LCG is doing it.

> BTW, the reported free space is useless for yet another
> reason: imagine
> there's 10 GB reported free, and 10 jobs read this information
> simultaneously (and they do, even more than 10), and duly start
> uploading a 2GB file each. Guess what will happen. Right, all
> will fail.

You certainly can't assume that you won't get any errors just because there
seems to be space, but if you know there isn't enough space there isn't much
point trying! Of course, it would be nice to have data management tools
which would automate it all.

>   Meanwhile, the SE GRIS will time out because the system will get
> overloaded with 10 multithreaded transfers, and 10 more jobs
> will still
> see the 2 GB free because this is what will be cached in the BDII. And
> so on. Ain't that cool.

Well, that's somewhat a different matter; it's certainly possible to
completely flatten an SE by doing many concurrent transfers, but that's true
whether it's full or empty. If the GRIS goes away I think the BDII will lose
it within a minute or two, so that's at least somewhat failsafe.

> So, we can block LHCb and they can block us. We're even ;-)

Well, except that they seem to be a bit less aggressive in their use of
space :)

> Nobody's perfect. A certain person here suggested to have data loss
> insurance :-) Smaller is the site, less compensation is to be paid.
> Profits from the insurance  company should finance purchase of more
> storage hardware. How's that? ;-)

It depends on the cost of recreating the data. You'd better not lose the raw
data from the detector ...

> Seriously, I would suggest to change the entire LCG SE model - and the
> information system schema.

Nothing major then :) I seem to remember when I first joined EDG I started
by trying to understand what an SE was supposed to be; I'm not sure I've
ever managed it ... part of the problem is that sites just think "we're
supposed to have an SE", so they put one in even if it's just a normal
machine with a 100 Gb disk. As you say, given good networking there's no
strong reason for that, sites can easily point their close SE somewhere else
- as NIKHEF have apparently done, although Sara isn't entirely a different
site.

> The disk space local to the site and necessary for its
> proper functioning should be renamed and treated as "cache"
> and must not be used for long-term data storage.

A local SE is never *necessary*, it's just a network bandwidth issue.

Stephen