Hi John,
We have 5x C6145's at Glasgow (so 10 nodes with 640 cores total) and if we had it to do again we probably wouldn't go for them. Some of the issues we've had with the boxes have been:
1) Lower HEPSPEC than stated (probably overcome by a move to SL6).
2) Issues with BIOS upgrades not working.
3) Unexplained shutdowns (possibly heat related).
4) Problems with PBS due to cpu density.
In general we've found they use a lot more power than other nodes we have for the HEPSPEC we get.
In a follow up purchase we actually went for Intel Sandybridge Xeons in a twin-squared chassis from Supermicro which gave us the same 128 cores in a 2U density but spread across 4 nodes rather than 2 (this is without a 10Gig interface though).
I know other sites haven't had the same number of problems we've had but that's the experience with them at Glasgow.
Gareth
On 3 Dec 2012, at 10:47, John Hill wrote:
> Hi all,
> I'm looking to add some CPUs to our Tier 2. I was considering the
> possibility of the C6145 (or similar) and I believe that other sites
> already have these. Hence I was interested in people's experiences. Are
> there unexpected gotchas from putting 48+ cores in a single box,
> expecially for a smaller Tier 2 site like ours, or is it all plain sailing?
> Thanks for any thoughts.
>
> John
|