On 4 August 2010 16:32, Alessandra Forti <[log in to unmask]> wrote:
> PS
>>
>> Fundamentally, it's because DPM only does round-robin balancing of
>> transactions between filesystems.
>> So, if you have one 10TB disk server with 1 filesystem (=10TB) and one
>> 30TB disk server with 1 filesystem (=30TB), then the first disk server
>> will fill up completely (on average) when the second is only 1/3 full.
>> In addition, this also means that if you have a mix of disk servers
>> with different io performance (and they all only have one fs on them),
>> then the more performant disk servers will be happy when the less
>> performant ones are melting (which is generally not a good use of
>> resource capacity).
>>
>>
>
> I understand that it is useful to do "mix and match" but even more then I
> would prefer to have the flexibility of creating fs the size I want to do
> the right matching for us would be good to have 20TB to match the previous
> fs.
No, I agree entirely. Hence our testing of the new 64bit e2fstools :D
>
>> Although, I do agree with you that ext4 (in fact, IIRC, ext* limits,
>> since it's the e2fsprogs tools which are currently limited to 32bit
>> here)
>
> the fs system itself is 48bit according to this
>
> https://ext4.wiki.kernel.org/index.php/Ext4_Howto#Bigger_File_System_and_File_Sizes
>
> last updated on the 9/7/2010.
>
> As I said might not make much difference but to me gives the impression of
> product not yet mature.
Well, you know that ext4 is "just" ext2 with various extensions, just
like ext3...
I believe Theodore T'so considers it a stopgap until btrfs is stable
and ready for production use.
Sam
>
> cheers
> alessandra
> cheers
> alessandra
>
>
>> limits to 16TB is unfortunately restrictive. As we also
>> discussed after the "official" meeting finished, there does seem to be
>> a 64bit version of e2fsprogs in existence, and we can test this to see
>> if it works for making suitably giant filesystems. (It looks, from the
>> linux filesystem mailing lists, as if it basically works for
>> filesystem creation, but there are various other e2fsprogs tools that
>> don't - online filesystem resize, for example).
>>
>> Sam
>>
>>
>>>
>>> At the moment looking at what I wrote, and without any additional number
>>> that demonstrates the performance gain, I'm leaning towards installing
>>> XFS
>>> on SL5.5 and see if there is any improvement.
>>>
>>> cheers
>>> alessandra
>>>
>>>
>>> [log in to unmask] wrote:
>>>
>>>>
>>>> Oops, this should have gone to the list, not to me!
>>>>
>>>>
>>>> Minutes already uploaded! (helps that my 11 o'clock meeting was
>>>> cancelled) Once again a very useful and productive session, I thought.
>>>> Lots of good stuff.
>>>>
>>>> http://storage.esc.rl.ac.uk/weekly/20100804-minutes.txt
>>>>
>>>> New actions include me volunteering more experiments and Matt to work
>>>> with Sam on testing T2K at Lancaster.
>>>>
>>>> So IMHO, the agenda for storage at GridPP could look like this:
>>>> 16.00-16.45 Experiments presenting (including T2K, perhaps 10 mins
>>>> each?)
>>>> 16.45-17.05 Sam reporting on AmDamJam and IC (with input from everyone
>>>> who went!)
>>>> 17.05-17.20 Me talking tasks, roadmap and stuff, we can discuss content.
>>>> 17.20-17.50 Discussion
>>>>
>>>> Cheers
>>>> --jens
>>>>
>>>>
>>>
>>> --
>>> The most effective way to do it, is to do it. (Amelia Earhart)
>>> Northgrid Tier2 Technical Coordinator
>>> http://www.hep.manchester.ac.uk/computing/tier2
>>>
>>>
>
> --
> The most effective way to do it, is to do it. (Amelia Earhart)
> Northgrid Tier2 Technical Coordinator
> http://www.hep.manchester.ac.uk/computing/tier2
>
>
|