On 14 Jun 2006, at 16:39, Jensen, J (Jens) wrote:
> Graeme Stewart wrote on 14 June 2006 15:06:
>
>> On 13 Jun 2006, at 16:27, Jensen, J (Jens) wrote:
>>
>>> Greig A Cowan wrote on 13 June 2006 14:16:
>>>>
>>>>> I agree. My point (if any) was that implementing the space API
>>>>> is a
>>>>> way to implement per-VO-quotas, plus it gives you some sort of
>>>>> advertised management of the space it's managing (ie tapeXdiskY or
>>>>> whatever it was). If experiments have asked for guaranteed
>>>>> reservations, then we can use that as a quota facility.
>>>>
>>>> So you are saying that experiments could ask for long-lived space
>>>> tokens
>>>> in order to guarantee themselves space? I was thinking that the
>>>> space
>>>> token would only be a short lived entity that would expire once the
>>>> request it was associated with had completed (i.e. the file had
>>>> been
>>>> copied into the space).
>>>>
>>>
>>> I understood from the new SRM 2.2 space model that that was how it
>>> worked: the VO manager talks to the admin, by phone possibly, and
>>> asks
>>> for space. (Caveat: just based on the summary Graeme gave last
>>> Wed.)
>>>
>>> If space is requested via the interface then it can be more dynamic.
>>>
>>> So we may have either, or both, types of space, depending on the
>>> implementation. DPM should support the dynamic stuff, whereas
>>> CASTOR
>>> will probably only support the "phone space".
>>>
>>> Cheers.
>>> --jens
>>
>> To my mind space reservation and quotaing are quite different. You
>> can't use a reserved space to do quotaing as you cannot force
>> incoming files to use a certain space token.
>
> Actually you can, because the space is associated with a specific path
> - the space token is not passed with the put (says Shaun - I still
> haven't had time to RTFM).
This is a castorism, it's not general to SRM v2.2, and it's not the
way that J-P will implement it in DPM - I spoke to him today.
If you look at the 2.2 prepareToPut is _does_ include a space token:
http://sdm.lbl.gov/srm-wg/doc/SRM.v2.2.html#_Toc138043958
And there's nothing in the WLCG clients MOU to indicate that WLCG
clients should not do this (https://srm.fnal.gov/twiki/pub/
WorkshopsAndConferences/GridStorageInterfacesWSAgenda/SRMLCG-MoU-
day2.doc)
>
>>
>> J-P was of the opinion that quotaing would be reasonably easy to
>> implement in DPM. I'm sure that he could do it in a couple of weeks -
>> but does he have those weeks?
>
> It's actually not entirely trivial because rfiod and the ftpd must
> check the TURL with some database before allowing the write, and to
> enforce the limit they get out of the database.
It could be that you would allow the write to happen if there was at
least the default file size left in the quota, which would allow a 1
file spill over if someone wrote a particularly large file, but
unless this was done maliciously it would be unlikely to be serious.
At least DPM contains this notion. I can ask j-p.
> Unless you block it at a lower level, but that's what we're trying to
> avoid - cleanest to return the error from the highest level.
>
>>
>> N.B. Space tokens can be very long lasting - reserving space on disk
>> for Atlas's current AOD set for the next 6 months. If DPM implements
>> genuine dynamic space tokens, as J-P intends, then shorter space
>> tokens could be issued for a user's analysis job.
>>
>
> I would think some spaces could implement quotas as part of the QoS,
> and it's up to the sysadmin to administrate. Of course, I'm just
> writing wishlists...
As if there wasn't enough to do!
cheers
g
--
Dr Graeme Stewart - http://wiki.gridpp.ac.uk/wiki/User:Graeme_stewart
GridPP DM Wiki - http://wiki.gridpp.ac.uk/wiki/Data_Management
|