We haven't documented this really, but the essence if as follows:
We share WNs and Batch server only, all the other components are
independent
- On CE:
USER_HOME_PREFIX=/grid/home # This path used on CE only
BATCH_SERVER=farm00.$MY_DOMAIN
- Pool account homedirs served via NFS (independent server for local
user homedirs)
- Local users accounts on LDAP, pool accounts created locally on WNs
with homedir path /grid/home
- TAR_WN served via NFS by a tar_wn_host
- The following settings on TAR_WN are possibly not optimal, but work:
INSTALL_ROOT=/path/to/install/glite-version-on-nfs-server
GLITE_EXTERNAL_ROOT=${INSTALL_ROOT}/external # Missing from
example file
GRID_ENV_LOCATION=${GLITE_EXTERNAL_ROOT}/etc/profile.d # Missing from
example file
On the WNs we have then links for "certificates" "vomsdir" and "grid-
env*" of the type:
/etc/profile.d/grid-env.csh -> /path/to/install/glite-version-on-nfs-
server/external/etc/profile.d/grid-env.csh
...
/etc/grid-security/certificates -> /path/to/install/glite-version-on-
nfs-serverr/external/etc/grid-security/certificates
/etc/grid-security/vomsdir -> /path/to/install/glite-version-on-nfs-
server/etc/grid-security/vomsdir
These work best if the path on the NFS server is fully explicit and
does NOT itself include a link of the type
/path/to/install/glite -> /path/to/install/glite-version-on-nfs-server
- On tar_wn_host we need:
WN_LIST=${INSTALL_ROOT}/glite/yaim/etc/wn-list.conf # This is used
for WN_TAR
This file is a one-liner including the FQ name for the tar_wn_host
- One queue per VO in batch server, independent queues for local
users. Can tune node assignment to queue in Torque/Maui if needed.
Fair shares, queue priorities, etc, all set in torque/maui on batch
server.
That's all I can think now off the top of my head. If you need more
details, let me know.
cheers,
Gianfranco
On 28 Oct 2009, at 15:16, Elena Korolkova wrote:
> Hi Gianfranco
>
> thanks a lot for the information. This is exatcly what we wanted to
> know.
>
> Do you have description somewhere on the web how your shared cluster
> works? In Sheffield, we want to share only WNs and have independent
> storage system for both clusters as well separate pbs servers. We
> would like to have some ideas how can we arrange this.
>
> Thanks a lot once more.
>
> Elena
>
> On Wed, 28 Oct 2009, Gianfranco Sciacca wrote:
>
>> Hi Elena,
>>
>> we use a shared area automounted as /grid/home on the CE and WNs,
>> we have /home anywhere else.
>>
>> In site-info.def on the CE we have:
>> USER_HOME_PREFIX=/grid/home # This path used on CE only
>>
>> cheers,
>> Gianfranco
>>
>>
>> On 28 Oct 2009, at 13:42, Elena Korolkova wrote:
>>
>>> Hello
>>> we are thinking about using resources (WNs) of our local cluster
>>> (T3) for the grid T2 cluster.
>>> What setup are people using for the shared clusters?
>>> In particular, does someone use something other than /home for
>>> user grid accounts?
>>> Your help is much appreciated
>>> Elena
>
> ____________________________________________________________________________
> Dr Elena Korolkova
> Email: [log in to unmask]
> Tel.: +44 (0)114 2223553
> Fax: +44 (0)114 2223555
> Department of Physics and Astronomy
> University of Sheffield
> Sheffield, S3 7RH, United Kingdom
--
Dr. Gianfranco Sciacca Tel: +44 (0)20 7679 3044
Dept of Physics and Astronomy Internal: 33044
University College London D15 - Physics Building
London WC1E 6BT
|