Unfortunately gLite "suggests" separate installation environments:
https://twiki.cern.ch/twiki/bin/view/LCG/GenericInstallGuide320#The_middleware_repositories
«Note that installation of several node types in the same physical host is not recommended. The repositories of each node type may not be synchronised for the same package and this can cause problems»
Our site is currently running for this (and other more questionable) reason each node type in a separate VM using a scheme that work-ish but I have learned to distrust (for various issues, essentially all of them legacy).
So I am looking for alternatives. Ideally we'd have a physical host with several node types on it; ideally I would have two physical hosts with *all* node types on each, for redundancy of some sort (hot/hot or more likely hot/warm) and in some cases load sharing. But I can see other problems with this (certificates and/or port conflicts, which I think are soluble).
But the repo conflicts mentioned above are a bit of a worry...
Overall we don't need loads of server nodes, around 6-8 probably (for 4-6 different node types). I am excluding from the discussion disk nodes here because they are a special case because of lots of disk slots, even if that could be accommodated with a SAS-cabled enclosure.
So I have been thinking about two possible replacements, one is to use "partitions" as in the Linux VServer project that I have previously used fairly happily:
http://en.wikipedia.org/wiki/Linux-VServer
The advantage of these are that they involve enormously lower overheads and bugs than many full-VM setups, and still allow completely distinct chrooted installation environments (to the point that different distros can be run in each, as long as they can share the same kernel instance, which most can, and is not an issue for GridPP as they would be all SL5 systems).
Also using smaller dedicated servers, for example something like this 4-nodes-in-a-box:
http://www.pcpro.co.uk/reviews/servers/364477/broadberry-cyberserve-x34-q104
or back-and-front rackable short-length X3400 or even Atom based servers. There are several available from most of the usual names as they make a lot of sense for network services servers (which is what most Grid server nodes sort of are), and there is an interesting spread of options in this page for example:
http://www.rackservers.com/Category.aspx?c=28
These also have the big advantage for me that they are easier to physically handle for a single person.
I have even been thinking of replacing the front end servers with laptops.
So have you got comments? In particular about:
* Any experiences of using VServer (or OpenVZ) for Grid deployment?
* What about Atom based server nodes? Except perhaps for CEs where X3400s
might be required?
|