On 17/02/2012 14:14, Christopher J.Walker wrote:
> On 17/02/12 12:50, Wahid Bhimji wrote:
>>
>> OK i can stand corrected . Interesting interpretation of read-only.
>>
>
> It's the interpretation that means "no writes", but yes.
>
> Lustre has the ability to deactivate OSTs - so no new objects are
> allocated to them, but existing objects can be read or modified.
>
> For taking a machine out of service, it's exactly what you want...
Is it? If you're taking the machine out of service then what about the
data that is still on it? I prefer the hdfs approach - switch it off, go
and do something else for an hour or so and then you can do what you
want with the server. That's what I call taking it out of service.
Duncan
> Chris
>
>> Wahid
>>
>> On 17 Feb 2012, at 12:32, Sam Skipsey wrote:
>>
>>>
>>>
>>> On 17 February 2012 12:28, Alessandra Forti<[log in to unmask]
>>> <mailto:[log in to unmask]>> wrote:
>>>
>>> Actually it seems that atlas could almost empty one of our servers
>>> while it was in readonly.
>>>
>>>
>>> Indeed, we have the same experience at Glasgow. RDONLY filesystems can
>>> be cleaned out by the ATLAS consistency tools, for example. (And, they
>>> should probably be RDONLY if they need consistency checking in the
>>> first place).
>>>
>>> Sam
>>>
>>>
>>> cheers
>>> alessandra
>>>
>>>
>>> On 17/02/2012 12:08, Wahid Bhimji wrote:
>>>>
>>>> I would say that setting read-only presumably also doesn't let
>>>> files get deleted.
>>>> So setting the weight to 0 is actually better as no new stuff
>>>> will go there but unneeded stuff can be deleted.
>>>>
>>>> Wahid
>>>>
>>>> On 17 Feb 2012, at 11:55, Sam Skipsey wrote:
>>>>
>>>>>
>>>>>
>>>>> On 17 February 2012 11:44, Duncan Rand
>>>>> <[log in to unmask]<mailto:[log in to unmask]>>
>>>>> wrote:
>>>>>
>>>>> Royal Holloway has being using the weighting system. Our
>>>>> interpretation is to use a number between 1 and 9, the
>>>>> higher the number indicating that more files will be copied in.
>>>>>
>>>>>
>>>>> It's more that the numbers are a *relative* weighting of the
>>>>> filesystem in question. (That is, you can use numbers greater
>>>>> than 9, but a) the underlying implementation would end up with a
>>>>> very long list of filesystems to iterate around, b) all this
>>>>> gives you is more "precision" in your weighting.) For example,
>>>>> if you have two filesystems, one with weighting 100 and one with
>>>>> weighting 200, the end result would be that fs 1 gets half as
>>>>> many files as fs2; the same result as giving them weightings of
>>>>> 1 and 2. The side-effect of the implementation is that the
>>>>> bigger weighting numbers would result in a filesystem list of
>>>>> length 300, rather than 3, which is probably not as efficient
>>>>> internally (DPM does not attempt to find common factors in order
>>>>> to compute an optimal representation of the actual weighting
>>>>> ratios).
>>>>>
>>>>>
>>>>> As Wahid suggests a zero apparently stops new files being
>>>>> copied in without having to mark a file system read only.
>>>>> This is very useful as it avoids reducing the apparent
>>>>> available space outside of the ATLAS space-tokens thereby
>>>>> solving Wahid's second bad experience:
>>>>>
>>>>> "2. marking a disk read only making sam tests fail".
>>>>>
>>>>>
>>>>> "Working around Wahid's second bad experience", really. The
>>>>> breaking of spacetokens and free space in the information system
>>>>> when marking filesystems read-only is a bug, and should not be
>>>>> considered "fixed" by having a way of avoiding marking
>>>>> filesystems read-only.
>>>>>
>>>>> Sam
>>>>>
>>>>>
>>>>> Duncan
>>>>>
>>>>>
>>>>> On 17/02/2012 11:25, Wahid Bhimji wrote:
>>>>>
>>>>> Hi
>>>>>
>>>>> Well just to say - I don't think there is anyway that
>>>>> it will break anything.Worst is an unexpected
>>>>> distribution of files.
>>>>>
>>>>> And for example setting 0 on a filesystem nearing
>>>>> capacity is an easy way of not getting it more full
>>>>> without needing to understand in detail the selection rules.
>>>>>
>>>>> cheers
>>>>>
>>>>> Wahid
>>>>>
>>>>> On 17 Feb 2012, at 11:15, John Bland wrote:
>>>>>
>>>>>
>>>>> Hi,
>>>>>
>>>>> Thanks, I only found out of date man pages. Checking
>>>>> again I've found the man pages that came with the
>>>>> package, 'usefully' in the non-standard
>>>>> /opt/lcg/share/man/ directory.
>>>>>
>>>>> The second link is very useful, it's not using
>>>>> weights in the way I was expecting it to, so it will
>>>>> take some more consideration in how to set them up.
>>>>>
>>>>> Being untested I'm slightly less keen to use it now
>>>>> ;0). I'll give it a try when I get time to keep an
>>>>> eye on it.
>>>>>
>>>>> John
>>>>>
>>>>> On 17/02/2012 11:06, Wahid Bhimji wrote:
>>>>>
>>>>>
>>>>> Hi John
>>>>>
>>>>> I think the only documentation is the man pages
>>>>>
>>>>> http://grid-deployment.web.cern.ch/grid-deployment/documentation/LFC_DPM/dpm/man1/dpm-modifyfs.1.html
>>>>>
>>>>> for dpm-modifyfs it only tells you
>>>>> weight:
>>>>> specifies the weight of the filesystem. This is
>>>>> used during the filesystem selection. The value
>>>>> must be positive. It is recommended to use a
>>>>> value lower than 10.
>>>>>
>>>>> but then the page for dpm-buildfsv
>>>>> http://grid-deployment.web.cern.ch/grid-deployment/documentation/LFC_DPM/dpm/man1/dpm-buildfsv.1.html
>>>>>
>>>>> gives you a bit more idea of what will happen if
>>>>> you use different values.
>>>>> I don't think we have much experience of what
>>>>> will happen in practise except for a very quick
>>>>> test with JP when he was developing. Some
>>>>> experience in production would be very
>>>>> interesting to hear about!
>>>>>
>>>>> Cheers
>>>>>
>>>>> Wahid
>>>>>
>>>>> On 17 Feb 2012, at 10:59, John Bland wrote:
>>>>>
>>>>> Hi,
>>>>>
>>>>> After our upgrade to DPM 1.8.2 I'm keen to
>>>>> investigate filesystem weights. We have a
>>>>>
>>>>> dpm-modifyfs --weight weight
>>>>>
>>>>> option now but no documentation to tell us
>>>>> what values it accepts and how the weights
>>>>> are calculated and used. Anyone have any
>>>>> links to some documentation (or just a one
>>>>> line "here's what the options are")?
>>>>>
>>>>> Thanks,
>>>>>
>>>>> John
>>>>>
>>>>> --
>>>>> John Bland
>>>>> [log in to unmask]
>>>>> <mailto:[log in to unmask]>
>>>>> System Administrator office: 220
>>>>> High Energy Physics Division tel (int):
>>>>> 42911
>>>>> Oliver Lodge Laboratory tel (ext):
>>>>> +44 (0)151 794 2911
>>>>> <tel:%2B44%20%280%29151%20794%202911>
>>>>> University of Liverpool
>>>>> http://www.liv.ac.uk/physics/hep/
>>>>> "I canna change the laws of physics, Captain!"
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> John Bland
>>>>> [log in to unmask]<mailto:[log in to unmask]>
>>>>> System Administrator office: 220
>>>>> High Energy Physics Division tel (int): 42911
>>>>> Oliver Lodge Laboratory tel (ext): +44
>>>>> (0)151 794 2911<tel:%2B44%20%280%29151%20794%202911>
>>>>> University of Liverpool
>>>>> http://www.liv.ac.uk/physics/hep/
>>>>> "I canna change the laws of physics, Captain!"
>>>>>
>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>>
>>>> The University of Edinburgh is a charitable body, registered in
>>>> Scotland, with registration number SC005336.
>>>
>>>
>>
>>
>>
>> The University of Edinburgh is a charitable body, registered in
>> Scotland, with registration number SC005336.
|
