Hi Owen,
thanks.
Just in case you want to add it to your set of patches to report. I've
attached my config_gip I don't think it is general enough to propose a
real patch because the SE section is only one and I don't know how they
want to integrate dcache,dpm and classic SE together. It gives anyway
a good idea of what I think needs to be changed for dcache accordingly to
Tier1 lcg-info.
Other changes I think should be done are:
1) Some glue schema fields are hardcoded in the script.
They should go into a configuration file/template that can be
safely edited, editing scripts is not a good practice.
example of fields:
GlueSAPolicyFileLifeTime: permanent
GlueSAPolicyMaxFileSize: 10000
GlueSAPolicyMinFileSize: 1
GlueSAPolicyMaxData: 100
GlueSAPolicyMaxNumFiles: 10
GlueSAPolicyMaxPinDuration: 10
GlueSAPolicyQuota: 00
GlueSAStateAvailableSpace: 1
2) the dynamic part of the ldif contained in
/opt/lcg/var/gip/tmp/lcg-info-dynamic-se.ldif.XXXX
doesn't get created (the file is 0 size)
I can't find what is missing right now and I'm a bit tired, tomorrow
I'll look better. I think that it is the wrapper that doesn't get called
or run by some other program. If anyone has any idea.... let me know.
cheers
alessandra
On Wed, 25 May 2005, Owen Synge wrote:
> Steve T gave me some more support on this issue, of publishing the
> incorrect information. It should help you all. I attached the
>
> /opt/lcg/var/gip/lcg-info-generic.conf
>
> earlier in the thread, which comes from the tier one D-Cache install.
>
> From Laurence field's home page some generic info on "gip" which is a
> very "Laurence" name for an application.
>
> http://lfield.home.cern.ch/lfield/cgi-bin/wiki.cgi?area=gip&page=documentation
>
> is the command for regenerating the information
>
> /opt/lcg/sbin/lcg-info-generic-config
> /opt/lcg/var/gip/lcg-info-generic.conf
>
> and should be done when ever /opt/lcg/var/gip/lcg-info-generic.conf is
> changed then the information provider uses
>
> su - edginfo -c '/opt/lcg/libexec/lcg-info-wrapper'
>
> To launch the script hierarchy.
>
> Regards
>
> Owen
>
>
> On Wed, 25 May 2005 14:41:16 +0100
> Alessandra Forti <[log in to unmask]> wrote:
>
>> Thanks better than parsing ldap. :)
>>
>> cheers
>> alessandra
>>
>>
>> On Wed, 25 May 2005, Owen Synge wrote:
>>
>>> On Wed, 25 May 2005 14:28:04 +0100
>>> Alessandra Forti <[log in to unmask]> wrote:
>>>
>>>> Hi,
>>>>
>>>> I think there is a bug in
>>>>
>>>> /opt/lcg/yaim/functions/config_gip
>>>>
>>>> line 366
>>>>
>>>> dn:
>>> GlueSARoot=$VO:${storage#${CE_CLOSE_SE1_ACCESS_POINT}/},GlueSEUnique
>>> ID=${SE_ > HOST},Mds-Vo-name=local,o=grid
>>>>
>>>> doesn't produce the desired output. I've replaced it with this.
>>>>
>>>> dn:
>>> GlueSARoot=$VO:${storage},GlueSEUniqueID=${SE_HOST},Mds-Vo-name=loca
>>> l,o=grid >
>>>> I'm looking for other things that might need change.
>>>>
>>>> cheers
>>>> alessandra
>>>>
>>> Great I shall add it to my patch collection, Attached is Steve's
>>> reference MDS provider written by hand for the tier 1
>>>
>>> Regards
>>>
>>> Owen S
>>>
>>> PS I went to a party for may day which had 4 people called Owen in
>>> one party an all time record for me.
>>>
>>>
>>
>> --
>> ********************************************
>> * Dr Alessandra Forti *
>> * Technical Coordinator - NorthGrid Tier2 *
>> * http://www.hep.man.ac.uk/u/aforti *
>> ********************************************
>
--
********************************************
* Dr Alessandra Forti *
* Technical Coordinator - NorthGrid Tier2 *
* http://www.hep.man.ac.uk/u/aforti *
********************************************
config_gip () {
INSTALL_ROOT=${INSTALL_ROOT:-/opt}
for x in CE_HOST CE_CLOSE_SE1_ACCESS_POINT SE_HOST RB_HOST PX_HOST GRIDICE_SERVER_HOST USERS_CONF; do
if [ "x`eval echo '$'$x`" = "x" ]; then
echo "\$$x not set"
return 1
fi
done
check_users_conf_format
outfile=$INSTALL_ROOT/lcg/var/gip/lcg-info-generic.conf
mkdir -p $INSTALL_ROOT/lcg/var/gip/tmp
chmod a+w -R ${INSTALL_ROOT}/lcg/var/gip/tmp/
cat <<EOF > $outfile
ldif_file=$INSTALL_ROOT/lcg/var/gip/lcg-info-static.ldif
generic_script=$INSTALL_ROOT/lcg/libexec/lcg-info-generic
wrapper_script=$INSTALL_ROOT/lcg/libexec/lcg-info-wrapper
temp_path=$INSTALL_ROOT/lcg/var/gip/tmp
template=$INSTALL_ROOT/lcg/etc/SiteInfo.template
template=$INSTALL_ROOT/lcg/etc/GlueCE.template
template=$INSTALL_ROOT/lcg/etc/GlueCESEBind.template
template=$INSTALL_ROOT/lcg/etc/GlueSE.template
template=$INSTALL_ROOT/lcg/etc/GlueService.template
# Common for all
GlueInformationServiceURL: ldap://`hostname -f`:2135/mds-vo-name=local,o=grid
EOF
if ( echo "${NODE_TYPE_LIST}" | grep CE > /dev/null ); then
for x in SITE_EMAIL SITE_NAME SITE_VERSION; do
if [ "x`eval echo '$'$x`" = "x" ]; then
echo "\$$x not set"
return 1
fi
done
cat << EOF >> $outfile
dn: siteName=$SITE_NAME,Mds-Vo-name=local,o=grid
sysAdminContact: $SITE_EMAIL
userSupportContact: $SITE_EMAIL
siteSecurityContact: $SITE_EMAIL
dataGridVersion: $SITE_VERSION
installationDate: `date +'%Y%m%d%H%M%SZ'`
EOF
for x in JOB_MANAGER CE_BATCH_SYS VOS QUEUES SE_HOST CE_BATCH_SYS CE_CPU_MODEL CE_CPU_VENDOR CE_CPU_SPEED CE_OS CE_OS_RELEASE CE_MINPHYSMEM CE_MINVIRTMEM CE_SMPSIZE CE_SI00 CE_SF00 CE_OUTBOUNDIP CE_INBOUNDIP CE_RUNTIMEENV CE_CLOSE_SE; do
if [ "x`eval echo '$'$x`" = "x" ]; then
echo "\$$x not set"
return 1
fi
done
for VO in $VOS; do
mkdir -p ${INSTALL_ROOT}/edg/var/info/$VO
# work out the sgm user for this VO
sgmuser=`awk -F: '$6=="sgm" && $5==tolower(virtorg){print $2}' virtorg=$VO ${USERS_CONF}`
sgmgroup=`id -g $sgmuser`
chown -R ${sgmuser}:${sgmgroup} ${INSTALL_ROOT}/edg/var/info/$VO
done
cat <<EOF >> $outfile
dynamic_script=${INSTALL_ROOT}/lcg/libexec/lcg-info-dynamic-ce
dynamic_script=${INSTALL_ROOT}/lcg/libexec/lcg-info-dynamic-software ${INSTALL_ROOT}/lcg/var/gip/lcg-info-generic.conf
# CE Information Provider
GlueCEHostingCluster: ${CE_HOST}
GlueCEInfoGatekeeperPort: 2119
GlueCEInfoHostName: ${CE_HOST}
GlueCEInfoLRMSType: $CE_BATCH_SYS
GlueCEInfoLRMSVersion: not defined
GlueCEInfoTotalCPUs: 0
GlueCEPolicyMaxCPUTime: 0
GlueCEPolicyMaxRunningJobs: 0
GlueCEPolicyMaxTotalJobs: 0
GlueCEPolicyMaxWallClockTime: 0
GlueCEPolicyPriority: 1
GlueCEStateEstimatedResponseTime: 0
GlueCEStateFreeCPUs: 0
GlueCEStateRunningJobs: 0
GlueCEStateStatus: Production
GlueCEStateTotalJobs: 0
GlueCEStateWaitingJobs: 0
GlueCEStateWorstResponseTime: 0
EOF
for x in $CE_RUNTIMEENV; do
echo "GlueHostApplicationSoftwareRunTimeEnvironment: $x" >> $outfile
done
cat <<EOF >> $outfile
GlueHostArchitectureSMPSize: $CE_SMPSIZE
GlueHostBenchmarkSF00: $CE_SF00
GlueHostBenchmarkSI00: $CE_SI00
GlueHostMainMemoryRAMSize: $CE_MINPHYSMEM
GlueHostMainMemoryVirtualSize: $CE_MINVIRTMEM
GlueHostNetworkAdapterInboundIP: $CE_INBOUNDIP
GlueHostNetworkAdapterOutboundIP: $CE_OUTBOUNDIP
GlueHostOperatingSystemName: $CE_OS
GlueHostOperatingSystemRelease: $CE_OS_RELEASE
GlueHostOperatingSystemVersion: 3
GlueHostProcessorClockSpeed: $CE_CPU_SPEED
GlueHostProcessorModel: $CE_CPU_MODEL
GlueHostProcessorVendor: $CE_CPU_VENDOR
dn: GlueClusterUniqueID=${CE_HOST}, mds-vo-name=local,o=grid
GlueClusterName: ${CE_HOST}
EOF
for QUEUE in $QUEUES; do
echo "GlueClusterService: ${CE_HOST}:2119/jobmanager-$JOB_MANAGER-$QUEUE" >> $outfile
done
for QUEUE in $QUEUES; do
echo "GlueForeignKey: GlueCEUniqueID=${CE_HOST}:2119/jobmanager-$JOB_MANAGER-$QUEUE" >> $outfile
done
cat << EOF >> $outfile
dn: GlueSubClusterUniqueID=${CE_HOST}, GlueClusterUniqueID=${CE_HOST}, mds-vo-name=local,o=grid
GlueChunkKey: GlueClusterUniqueID=${CE_HOST}
GlueSubClusterName: ${CE_HOST}
EOF
for QUEUE in $QUEUES; do
cat <<EOF >> $outfile
dn: GlueCEUniqueID=${CE_HOST}:2119/jobmanager-$JOB_MANAGER-$QUEUE, mds-vo-name=local,o=grid
GlueCEName: $QUEUE
GlueForeignKey: GlueClusterUniqueID=${CE_HOST}
EOF
for VO in `echo $VOS | tr [:lower:] [:upper:]`; do
for VO_QUEUE in `eval echo '$'VO_${VO}_QUEUES`; do
if [ "${QUEUE}" = "${VO_QUEUE}" ]; then
echo "GlueCEAccessControlBaseRule: VO:`echo $VO | tr [:upper:] [:lower:]`" >> $outfile
fi
done
done
done
for SE in $CE_CLOSE_SE; do
if [ "x`eval echo '$'CE_CLOSE_${SE}_HOST`" = "x" ]; then
echo "\$CE_CLOSE_${SE}_HOST not set"
return 1
fi
if [ "x`eval echo '$'CE_CLOSE_${SE}_ACCESS_POINT`" = "x" ]; then
echo "\$CE_CLOSE_${SE}_ACCESS_POINT not set"
return 1
fi
done
for SE in $CE_CLOSE_SE; do
for QUEUE in $QUEUES; do
cat <<EOF >> $outfile
dn: GlueCESEBindSEUniqueID=`eval echo '$'CE_CLOSE_${SE}_HOST`, GlueCESEBindGroupCEUniqueID=${CE_HOST}:2119/jobmanager-$JOB_MANAGER-$QUEUE, mds-vo-name=local,o=grid
GlueCESEBindCEAccesspoint: `eval echo '$'CE_CLOSE_${SE}_ACCESS_POINT`
GlueCESEBindCEUniqueID: ${CE_HOST}:2119/jobmanager-$JOB_MANAGER-$QUEUE
dn: GlueCESEBindGroupCEUniqueID=${CE_HOST}:2119/jobmanager-$JOB_MANAGER-$QUEUE, mds-vo-name=local,o=grid
GlueCESEBindGroupSEUniqueID: `eval echo '$'CE_CLOSE_${SE}_HOST`
EOF
done
done
fi #endif for CE_HOST
if [ "$GRIDICE_SERVER_HOST" = "`hostname -f`" ]; then
for x in VOS SITE_NAME SITE_EMAIL; do
if [ "x`eval echo '$'$x`" = "x" ]; then
echo "\$$x not set"
return 1
fi
done
cat <<EOF >> $outfile
dn: GlueServiceURI=ldap://${GRIDICE_SERVER_HOST}:2136,Mds-vo-name=local,o=grid
GlueServiceAccessPointURL: ldap://${GRIDICE_SERVER_HOST}:2136/mds-vo-name=local,o=grid
GlueServiceType: gridice
GlueServicePrimaryOwnerName: LCG
GlueServicePrimaryOwnerContact: $SITE_EMAIL
GlueServiceHostingOrganization: $SITE_NAME
GlueServiceMajorVersion: 1
GlueServiceMinorVersion: 1
GlueServicePatchVersion: 1
GlueServiceStatus: running
GlueServiceInformationServiceURL: ldap://${GRIDICE_SERVER_HOST}:2135/mds-vo-name=local,o=grid
EOF
for VO in $VOS; do
echo "GlueServiceAccessControlRule: $VO" >> $outfile
done
echo >> $outfile
FMON='--fmon=yes'
fi #endif for GRIDICE_SERVER_HOST
if ( echo "${NODE_TYPE_LIST}" | grep PX > /dev/null ); then
for x in GRID_TRUSTED_BROKERS SITE_EMAIL SITE_NAME; do
if [ "x`eval echo '$'$x`" = "x" ]; then
echo "\$$x not set"
return 1
fi
done
cat << EOF >> $outfile
dn: GlueServiceURI= ${PX_HOST}:7512,Mds-vo-name=local,o=grid
GlueServiceAccessPointURL: ${PX_HOST}:7512
GlueServiceType: MyProxy
GlueServicePrimaryOwnerName: LCG
GlueServicePrimaryOwnerContact: $SITE_EMAIL
GlueServiceHostingOrganization: $SITE_NAME
GlueServiceMajorVersion: 1
GlueServiceMinorVersion: 1
GlueServicePatchVersion: 1
GlueServiceStatus: running
GlueServiceInformationServiceURL: ldap://${PX_HOST}:2135/mds-vo-name=local,o=grid
EOF
split_quoted_variable $GRID_TRUSTED_BROKERS | while read x; do
echo "GlueServiceAccessControlRule: $x" >> $outfile
done
echo "" >> $outfile
fi #endif for PX_HOST
if ( echo "${NODE_TYPE_LIST}" | grep RB > /dev/null ); then
for x in VOS SITE_EMAIL SITE_NAME; do
if [ "x`eval echo '$'$x`" = "x" ]; then
echo "\$$x not set"
return 1
fi
done
cat <<EOF >> $outfile
dn: GlueServiceURI=gram://${RB_HOST}:7772,Mds-vo-name=local,o=grid
GlueServiceAccessPointURL: gram://${RB_HOST}:7772
GlueServiceType: ResourceBroker
GlueServicePrimaryOwnerName: LCG
GlueServicePrimaryOwnerContact: $SITE_EMAIL
GlueServiceHostingOrganization: $SITE_NAME
GlueServiceMajorVersion: 1
GlueServiceMinorVersion: 1
GlueServicePatchVersion: 1
GlueServiceInformationServiceURL: ldap://${RB_HOST}:2135/mds-vo-name=local,o=grid
GlueServiceStatus: running
EOF
for VO in $VOS; do
echo "GlueServiceAccessControlRule: $VO" >> $outfile
done
echo >> $outfile
fi #endif for RB_HOST
if ( echo "${NODE_TYPE_LIST}" | grep LFC > /dev/null ); then
for x in VOS SITE_EMAIL SITE_NAME BDII_HOST LFC_HOST; do
if [ "x`eval echo '$'$x`" = "x" ]; then
echo "\$$x not set"
return 1
fi
done
cat <<EOF >> $outfile
dn: GlueServiceURI=${LFC_HOST},mds-vo-name=local,o=grid
objectClass: GlueTop
objectClass: GlueService
objectClass: GlueSchemaVersion
GlueServiceURI: $LFC_HOST
GlueServiceAccessPointURL: $LFC_HOST
GlueServiceType: lcg-file-catalog
GlueServicePrimaryOwnerName: LCG
GlueServicePrimaryOwnerContact: mailto:${SITE_EMAIL}
GlueServiceHostingOrganization: ${SITE_NAME}
GlueServiceMajorVersion: 1
GlueServiceMinorVersion: 1
GlueServicePatchVersion: 1
EOF
for VO in $VOS; do
echo "GlueServiceAccessControlRule: $VO" >> $outfile
done
cat <<EOF >> $outfile
GlueServiceInformationServiceURL: MDS2GRIS:ldap://${BDII_HOST}:2170/mds-vo-name=local,mds-vo-name=${SITE_NAME},mds-vo-name=local,o=grid
GlueServiceStatus: running
GlueSchemaVersionMajor: 1
GlueSchemaVersionMinor: 1
EOF
fi # end of LFC
if ( echo "${NODE_TYPE_LIST}" | grep SE > /dev/null ); then
for x in VOS SE_HOST SITE_NAME SE_TYPE; do
if [ "x`eval echo '$'$x`" = "x" ]; then
echo "\$$x not set"
return 1
fi
done
for VO in `echo $VOS | tr [:lower:] [:upper:]`; do
if [ "x`eval echo '$'VO_${VO}_STORAGE_DIR`" = "x" ]; then
echo "\$VO_${VO}_STORAGE_DIR not set"
return 1
fi
done
# dynamic_script points to the script generated by config_info_dynamic_se<SE_TYPE>
echo "dynamic_script=${INSTALL_ROOT}/lcg/libexec/lcg-info-dynamic-se" >> $outfile
echo >> $outfile # Empty line to separate it form published info
cat <<EOF >> $outfile
GlueSAPolicyFileLifeTime: volatile
GlueSAPolicyMaxFileSize: 10000
GlueSAPolicyMinFileSize: 1
GlueSAPolicyMaxData: 100
GlueSAPolicyMaxNumFiles: 10
GlueSAPolicyMaxPinDuration: 10
GlueSAPolicyQuota: 00
GlueSAStateAvailableSpace: 1
GlueSAStateUsedSpace: 1
dn: GlueSEUniqueID=${SE_HOST},mds-vo-name=local,o=grid
GlueSEName: $SITE_NAME:${SE_TYPE}
GlueSEType: ${SE_TYPE}
GlueSEPort: 8443
GlueForeignKey: GlueSLUniqueID=${SE_HOST}
dn: GlueSLUniqueID=${SE_HOST},Mds-Vo-name=local,o=grid
GlueSLUniqueID: ${SE_HOST}
GlueSLName: ${SITE_NAME}
GlueSLArchitectureType: ${SE_TYPE}
GlueForeignKey: GlueSEUniqueID=${SE_HOST}
dn: GlueSEAccessProtocolType=gsiftp, GlueSEUniqueID=${SE_HOST},Mds-Vo-name=local,o=grid
GlueChunkKey: GlueSEUniqueID=${SE_HOST}
GlueSEAccessProtocolPort: 2811
GlueSEAccessProtocolSupportedSecurity: GSI
dn: GlueSEAccessProtocolType=gsidcap, GlueSEUniqueID=${SE_HOST},Mds-Vo-name=local,o=grid
GlueChunkKey: GlueSEUniqueID=${SE_HOST}
GlueSEAccessProtocolPort: 22128
GlueSEAccessProtocolSupportedSecurity: GSI
dn: GlueServiceURI=httpg://${SE_HOST}:8443/srm/managerv1,mds-vo-name=local,o=grid
GlueServiceURI: httpg://${SE_HOST}:8443/srm/managerv1
GlueServiceType: srm_v1
GlueServiceAccessPointURL: httpg://${SE_HOST}:8443/srm/managerv1
GlueServicePrimaryOwnerName: LCG
GlueServicePrimaryOwnerContact: mailto:${SITE_EMAIL}
GlueServiceHostingOrganization: ${SITE_NAME}
GlueServiceMajorVersion: 1
GlueServiceMinorVersion: 1
GlueServicePatchVersion: 1
GlueServiceAccessControlRule: cms, dteam, atlas, lhcb, babar, dzero
GlueServiceInformationServiceURL: MDS2GRIS:ldap://${SE_HOST}:2135/mds-vo-name=local,o=grid
GlueServiceStatus: running
EOF
for VO in $VOS; do
# Find the storage directory for this VO
storage=$( eval echo '$'VO_`echo ${VO} | tr '[:lower:]' '[:upper:]'`_STORAGE_DIR )
cat <<EOF >> $outfile
dn: GlueSARoot=$VO:${storage},GlueSEUniqueID=${SE_HOST},Mds-Vo-name=local,o=grid
GlueChunkKey: GlueSEUniqueID=${SE_HOST}
GlueSAAccessControlBaseRule: $VO
GlueSAUniqueID: $VO
EOF
done
fi #endif for SE_HOST
$INSTALL_ROOT/lcg/sbin/lcg-info-generic-config $INSTALL_ROOT/lcg/var/gip/lcg-info-generic.conf
cat << EOT > $INSTALL_ROOT/globus/libexec/edg.info
#!/bin/bash
#
# info-globus-ldif.sh
#
#Configures information providers for MDS
#
cat << EOF
dn: Mds-Vo-name=local,o=grid
objectclass: GlobusTop
objectclass: GlobusActiveObject
objectclass: GlobusActiveSearch
type: exec
path: $INSTALL_ROOT/lcg/libexec
base: lcg-info-wrapper
args:
cachetime: 60
timelimit: 20
sizelimit: 250
EOF
EOT
chmod a+x $INSTALL_ROOT/globus/libexec/edg.info
cat << EOT > $INSTALL_ROOT/globus/libexec/edg.schemalist
#!/bin/bash
cat <<EOF
$INSTALL_ROOT/globus/etc/openldap/schema/core.schema
$INSTALL_ROOT/globus/etc/grid-info-resource.schema
$INSTALL_ROOT/edg/share/info/schema/edg-core.schema
$INSTALL_ROOT/edg/share/info/schema/Glue-CE.schema
$INSTALL_ROOT/edg/share/info/schema/Glue-CESEBind.schema
$INSTALL_ROOT/edg/share/info/schema/Glue-SE.schema
$INSTALL_ROOT/edg/share/info/schema/edg-SI.schema
$INSTALL_ROOT/edg/share/info/schema/edg-network.schema
EOF
EOT
chmod a+x $INSTALL_ROOT/globus/libexec/edg.schemalist
#Turn on Gin for the GIP and maybe FMON
export RGMA_HOME=${INSTALL_ROOT}/edg
${INSTALL_ROOT}/edg/share/rgma-gin/scripts/rgma-gin-config.py --gip=yes ${FMON}
/sbin/chkconfig edg-rgma-gin on
/etc/rc.d/init.d/edg-rgma-gin restart
# Configure the dynamic plugin appropriate for the batch sys
case "$CE_BATCH_SYS" in
condor|CONDOR) plugin="${INSTALL_ROOT}/lcg/libexec/lcg-info-dynamic-condor /opt/condor/bin/ ${INSTALL_ROOT}/lcg/var/gip/lcg-info-generic.conf";;
lsf|LSF) plugin="${INSTALL_ROOT}/lcg/libexec/lcg-info-dynamic-lsf /usr/local/lsf/bin/ ${INSTALL_ROOT}/lcg/var/gip/lcg-info-generic.conf";;
*) plugin="${INSTALL_ROOT}/lcg/libexec/lcg-info-dynamic-pbs ${INSTALL_ROOT}/lcg/var/gip/lcg-info-generic.conf";;
esac
cat << EOF > ${INSTALL_ROOT}/lcg/libexec/lcg-info-dynamic-ce
#!/bin/sh
$plugin
EOF
chmod +x ${INSTALL_ROOT}/lcg/libexec/lcg-info-dynamic-ce
return 0
}
|
