Hi Wahid,
Sorry for not replying sooner, since your work here touches on something I've
been involved with.
First off, have you seen the Installed Capacity document:
https://twiki.cern.ch/twiki/pub/LCG/WLCGCommonComputingReadinessChallenges/WLCG_GlueSchemaUsage-1.8.pdf
This is the document describes how to do storage accounting: how sites should
publish information to allow storage accounting and how people can work out
what the storage is being used per-VO.
On Tuesday 15 December 2009 18:17:02 Wahid Bhimji wrote:
> The main difference, apart from some fixes and some of the things people
> wished for, is to include all entries from the BDII GlueSA info whether
> they have matching spacetokens (in VOInfo) or not.
In case this isn't obvious: a child VOInfo object advertises the ability of
some end-user group (usually a VO) to *write* into the storage capacity
represented by an SA. An SA can have zero or more VOInfo objects; an SA with
zero child VOInfo objects represents some capacity that people cannot write
into (e.g., staging space for accessing files from tape).
The SA object's ACBR attributes represents the ability of some end-users
(usually VOs) to use the storage capacity in some fashion: for reading,
writing, staging from tape, etc. So, if an SA object has a child VOInfo
object with some VO mentioned in the VOInfo.ACBR then the parent SA must also
mention that VO in the SA.ACBR.
> This shows space usage for those VOs that do not use spacetokens, or
> those like CMS, where some sites do and other don't.
US-CMS don't use space tokens, so that will likely account for some of the
sites that don't publish any space-tokens.
> I have also moved to also using "GlueSAStateAvailableSpace" instead of
> "GlueSAFreeOnlineSize" as that seems to be more reliably reported on.
According to Installed Capacity, all SA objects MUST (RFC-2119) publish
GlueSAFreeOnlineSize. Feel free to submit a GGUS ticket to any site that is
publishing an SA that doesn't :-)
GlueSAStateAvailableSpace is marked deprecated in Glue v1.3, but is required
in Installed Capacity. For the time being, sites MUST (RFC-2119 again)
publish this attribute, but this may change in the future.
> Unfortunately, however there seems to be no reliable recipe of which
> variables (or method srm v. bdii) to use for all sites and all VOs.
This is sadly true, but only because some sites are not yet adhering to the
Installed Capacity document.
[...]
> general way to get space usage for stuff that does not advertise iteself
> in spacetokens.
The general method for determining storage usage per VO is with the installed-
capacity property. See the above document for the details but, in brief, one
should look for the GlueSACapability that matches InstalledOnlineCapacity (and
InstalledNearlineCapacity). All SA objects should have one (again, feel free
to ticket any site that publishes a SA without an InstalledOnlineCapacity
attribute).
An InstalledOnlineCapacity of zero means that SA must not be included in any
space accounting. The remaining SAs (those with non-zero
InstalledOnlineCapacity) form a covering set of the storage in the SE.
You can take InstalledOnlineCapacity (IOC) to mean the total size of the SA,
(GlueSA)TotalOnlineSize to be the total available size that is currently
available, (GlueSA)UsedOnlineSize as the amount that is currently being used.
In general the following relationships hold:
IOC >= TotalOnlineSize >= UsedOnlineSize
TotalOnlineSize >= FreeOnlineSize
TotalOnlineSize = UsedOnlineSize + FreeOnlineSize
Most SAs will have an IOC with the same value as TotalOnlineSize. They differ
only when some of the underlying hardware is currently broken or off-line.
ReservedOnlineSize is roughly like IOC, but has some more complex semantics,
for backward comparability with existing clients. I'd recommend ignoring
ReservedOnlineSize and use IOC.
The accounting should be against the VOs listed in the SA object's ACBR
attributes: if there's more than one VO then that SA is shared between those
VOs. There's no definition on how to do this: a simple (naive?) approach would
be to divide the SA equally between the listed VOs; however, if one VO is
DTEAM and the other is ATLAS then this will likely under-report storage for
ATLAS.
HTH,
Paul.
|