If we need to provide cloud support how will this affect hardware?
If there is a demand for VMs, having more memory might be beneficial.
dan
On 18/11/13 20:55, Alessandra Forti wrote:
> On 18/11/2013 20:42, Wahid Bhimji wrote:
>> Sorry after I sent this I realised I shouldn't have said …
>>> guess you can save money on WN disk so thats a pro.
>> with the current atlas situation - even if you are set up for direct
>> access a lot of jobs (those using transforms) will still copy to the WN..
> I know there is no plan for production to move to direct IO.
>
>> (unless you have a posixy filesystem like qm's lustre).
>> So given we have no idea how long that will last, you probably can't
>> really change the wn disk situation now anyway (Though ecdf don't have
>> striped disk - though thats not our choice).
>
>>
>> Wahid
>>
>> On 18 Nov 2013, at 20:36, Wahid Bhimji <[log in to unmask]>
>> wrote:
>>
>>> On 18 Nov 2013, at 20:15, Alessandra Forti <[log in to unmask]>
>>> wrote:
>>>
>>>> On 18/11/2013 19:46, Wahid Bhimji wrote:
>>>>> On 18 Nov 2013, at 17:37, Alessandra Forti
>>>>> <[log in to unmask]> wrote:
>>>>>
>>>>>> On 18/11/2013 17:14, Ewan MacMahon wrote:
>>>>>>>> -----Original Message-----
>>>>>>>> From: Testbed Support for GridPP member institutes [mailto:TB-
>>>>>>>> [log in to unmask]] On Behalf Of Wahid Bhimji
>>>>>>>>
>>>>>>>> 1) Many sites have previously prefered to just have the loaded
>>>>>>>> server
>>>>>>>> nodes (prev R510s) with no expansion arrays. So perhaps there
>>>>>>>> should be
>>>>>>>> an option to have that configuration.
>>>>>>>> (at Edinburgh we prefer the option with the arrays though)
>>>>>>>>
>>>>>>> We've preferred the servers at Oxford on the grounds that they cost
>>>>>>> virtually the same (IIRC, at one point an R510 was actually
>>>>>>> marginally
>>>>>>> cheaper from the portal than the equivalent disk array once you
>>>>>>> factored
>>>>>>> in the RAID controller needed to use the array), and have all
>>>>>>> sorts of
>>>>>>> minor advantages.
>>>>>>>
>>>>>>>> 3) 64G RAM is quite a lot - but maybe it doesn't really change
>>>>>>>> the cost.
>>>>>>>>
>>>>>>> I'd say it's actually a little short for a box of this spec - our
>>>>>>> recent
>>>>>>> machines have been 30TB usable with either 24 or 32GB of memory, so
>>>>>>> roughly speaking, on the order of 1GB of RAM per 1TB of disk. Dave's
>>>>>>> example spec would have 120TB of disk, so with 64GB of RAM it's
>>>>>>> about
>>>>>>> half that ratio. On DPM disk servers, the RAM is basically all
>>>>>>> read cache,
>>>>>>> so it's nice to have.
>>>>>> Manchester is getting away with much less than this, but direct IO
>>>>>> might change things. Wahid never sent his pros and cons.
>>>>>>
>>>>> Well when I was referring to pros and cons that was more about
>>>>> current atlas configuration issues.
>>>> Well when I asked you I specified it would have been useful for the
>>>> procurement.
>>> You did - but I hadn't really thought about it from that perspective.
>>> I guess you can save money on WN disk so thats a pro. Not sure that
>>> there is a con given I think the current storage hardware we have is
>>> up to it. Anyway ATLAS analysis people have stated desires for people
>>> to support direct IO as that it makes for reasonable length jobs so
>>> its almost a requirement.
>>>
>>>>> Longer term I think theres only pros - the main one being it allows
>>>>> for much longer analysis jobs. Increased core counts is just going
>>>>> to mean more pressure on WN disk(s)….
>>>>> But people are doing direct access now with their current hardware
>>>>> so I don't think it needs any radical changes.
>>>> which sites? what configuration? Manchester so far could get away
>>>> with 16GB/60TB on most machines. As I said direct IO might change this.
>>>>
>>> At least Glasgow; Oxford; Liverpool ; ECDF - I don't know the other
>>> sites specs but we have around 24 G / 60 T .
>>> But I agree more might be better (even with copy.) Actually now I
>>> realise that of course the double size disk is already halving the
>>> ratio- so I completely withdraw my point 3 ..
>>>
>>> Wahid
>>>
>>>
>>>> cheers
>>>> alessandra
>>>>> On the RAM - its good point, Ewan that more the merrier really -
>>>>> maybe I should get some more for our current disk servers.
>>>>>
>>>>> Wahid
>>>>>
>>>>>
>>>>>
>>>>>> cheers
>>>>>> alessandra
>>>>>>>> 4) Disk options: just looking at the dell normal prices - 3TB
>>>>>>>> sata is
>>>>>>>> listed at 386 while 4 TB NL sas is 605. which is quite a lot
>>>>>>>> more. Anyway
>>>>>>>> we maybe need to double check which offers the best value.
>>>>>>>>
>>>>>>> Indeed; our approach has been that this choice is up to the
>>>>>>> vendor to
>>>>>>> decide which approach is the cheapest overall, but based on what
>>>>>>> we've
>>>>>>> seen so far (that is, indicative hints, not formal tender
>>>>>>> responses) it
>>>>>>> does seem that having more machines with 3TB disks works our
>>>>>>> slightly ahead
>>>>>>> of a fewer boxes with 4TB disks option.
>>>>>>>
>>>>>>>> 6) Also warranty can be expensive and ought to be specified e.g.
>>>>>>>> 4 yr pro
>>>>>>>> support.
>>>>>>>>
>>>>>>> We've been buying five year warranties on everything for a while
>>>>>>> - it adds
>>>>>>> trivially to the cost, and avoids the risk of having to write a
>>>>>>> machine off
>>>>>>> prematurely for want of something trivial like a PSU. And given
>>>>>>> recent trends
>>>>>>> in both our kit retirement schedule, and the (lack of) increase
>>>>>>> in disk
>>>>>>> capacities, I would confidently expect storage servers to have a
>>>>>>> life of at
>>>>>>> least five years, probably more like 7-8.
>>>>>>>
>>>>>>> All that said, my main feeling about this is that it would have
>>>>>>> been a good
>>>>>>> idea if we'd done it earlier, but it's too late now given that
>>>>>>> most of us
>>>>>>> have had the spend shifted from a planned FY14 to
>>>>>>> FY-blind-panic-do-it-now.
>>>>>>>
>>>>>>> Ewan
>>>>>> --
>>>>>> Facts aren't facts if they come from the wrong people. (Paul Krugman)
>>>>>>
>>>>
>>>> --
>>>> Facts aren't facts if they come from the wrong people. (Paul Krugman)
>>>>
>>
>
>
--
* Dr Daniel Traynor, Grid system administrator
* Physics, QMUL, London, Tel +44(0)2078826560
|