On Wed, 15 Feb 2006 16:26:01 +0000
Matt Doidge <[log in to unmask]> wrote:

> Heya dudes,
> Our Information publishing problems are all fixed -but how we did it
> is another story that Owen and I are trying to sort out. We still need
> to do some tidying to sort out our VO's. We've got the latest dcache
> version with postgres, and was wondering what's the formula these days
> for creating new VO databases/binding them to pools? I can't find out
> how to do this on the web anywhere, was wondering if you chaps know
> anymore then I (it wouldn't be hard!).
> 
> cheers,
> matt

Ive been looking for this today also, as I am having problems with my 
information publisher too. I asked Derek from some help and got the
following

<quote>

Typically when I make changes to this setup, I'm doing it live through
the admin interface but its the same commands.

Two main things I do are adding pools and adding vos:

1. Adding a pool
When the new pool has been started up it will register itself here and
get added to the default pgroup. Since we want to partition the space I
removed the link that connect the default pool group to any selection
unit, so nothing ever uses the default pool group. When we decide which
vo is getting this space I do 
    psu addto pgroup <vo>-pgroup <poolname> 
The <vo>-pgroup is my convention, no need to follow it, in fact if you
do the info provider stuff won't ever see the space :-)

2. Adding a vo
Assuming the correct tags have been set up on the
/pnfs/gridpp.rl.ac.uk/data/<vo> directory, I do 
  psu create unit -store <vo>:<vo>@osm
Things are different for tape, but not very. I then create a ugroup to
store that unit :
  psu create ugroup <vo>
  psu addto ugroup <vo> <vo>:<vo>@osm
After that I create a pool group for their disk pools and add any pools
they're getting
  psu create pgroup <vo>-pgroup
  psu create pgroup <vo>-pgroup <poolname>
Then we need to link the <vo> ugroup to the <vo>-pgroup
  psu create link <vo>-link <vo>
  psu set link <vo>-link -readpref=10 -writepref=10 -cachepref=10
  psu add link <vo>-link <vo>-pgroup

---------------

So if you do the above then when new pools are added they won't get used
automatically, so when installing a new dcache system using YAIM, there
would need to be some manual post-install configuration, but if you have
the right name for the pgroups then the info provider would pick up
these changes automatically.

Or you could create link from the <vo> ugroups to the default pgroup and
everything get spread over all pools and working without any manual
intervention, however with the info provider plugin as provided, you're
not going to get any per vo information.

If we could get the info provider plugin source then perhaps we could
extend it to check a config file for pool group to vo mappings and spit
out the per vo info based on that, rather than relying on the pool group
having the same name as the vo.

Derek

</quote>

then he attached the PoolManager config file


/opt/d-cache/config/PoolManager.conf

which is cut n paste at the end of this file.

I have been working from two main sources of information for the new
info provider.

http://www.dcache.org/manuals/Book/cf-pm-psu.shtml
http://www.dcache.org/manuals/Book/cb-general-lcg.shtml

and


http://wiki.gridpp.ac.uk/wiki/Scotgrid_LCG_2.7_Pre-Release_Testing#Information_system

and the included data from Derek.

I do hope this helps you out, but as I see it,

I am thinking that Chris Brew has a nice idea of adding all pool nodes
in a pool group and then giving all VO's access to the pool group. This
is effectively the same as not setting up pools but with advantages in
management via D-Cache and the ability to use the new information
provider. I am unsure if the new information provider will get in to the
next release of YAIM as there are some issues that require hand
configuration and I am not convinced I can be generic enough with YAIM
without adding to much complexity, (I realise now I am near finished
with the job). I will see if Chris has any suggestions tomorrow and
hopefully I can put the new information system into YAIM before end of
business tomorrow and it will then meet my deadline for YAIM patches.

I am begging to think that this information provider could do with
taking more config from D-Cache and less from config files but it is the
newest component of D-Cache.

Regards

Owen








#
# Setup of PoolManager (diskCacheV111.poolManager.PoolManagerV5) at Tue
# Jan 31 18:13:36 GMT 2006
#
set pool decision  -spacecostfactor=1.0 -cpucostfactor=1.0
set timeout pool 15
#
#
# Printed by diskCacheV111.poolManager.PoolSelectionUnitV1 at Tue Jan 31
# 18:13:36 GMT 2006
#
#
#
# The units ...
#
psu create unit -store  lhcb:tape@osm
psu create unit -store  ilc:ilc@osm
psu create unit -store  dteam:tape@osm
psu create unit -store  *@*
psu create unit -store  pheno:pheno@osm
psu create unit -store  magic:magic@osm
psu create unit -store  atlas:tape@osm
psu create unit -store  atlas:atlas@osm
psu create unit -store  cms:tape@osm
psu create unit -store  lhcb:lhcb@osm
psu create unit -store  zeus:zeus@osm
psu create unit -net    0.0.0.0/255.255.255.255
psu create unit -store  dteam:dteam@osm
psu create unit -net    0.0.0.0/0.0.0.0
psu create unit -store  hone:hone@osm
psu create unit -store  babar:babar@osm
psu create unit -store  cms:cms@osm
psu create unit -store  esr:esr@osm
psu create unit -store  bio:bio@osm
psu create unit -store  t2k:t2k@osm
#
# The unit Groups ...
#
psu create ugroup lhcb
psu addto ugroup lhcb lhcb:lhcb@osm
psu create ugroup zeus
psu addto ugroup zeus zeus:zeus@osm
psu create ugroup cms-tape
psu addto ugroup cms-tape cms:tape@osm
psu create ugroup babar
psu addto ugroup babar babar:babar@osm
psu create ugroup cms
psu addto ugroup cms cms:cms@osm
psu create ugroup pheno
psu addto ugroup pheno pheno:pheno@osm
psu create ugroup magic
psu addto ugroup magic magic:magic@osm
psu create ugroup dteam-tape
psu addto ugroup dteam-tape dteam:tape@osm
psu create ugroup atlas
psu addto ugroup atlas atlas:atlas@osm
psu create ugroup world-net
psu addto ugroup world-net 0.0.0.0/0.0.0.0
psu create ugroup bio
psu addto ugroup bio bio:bio@osm
psu create ugroup hone
psu addto ugroup hone hone:hone@osm
psu create ugroup lhcb-tape
psu addto ugroup lhcb-tape lhcb:tape@osm
psu create ugroup atlas-tape
psu addto ugroup atlas-tape atlas:tape@osm
psu create ugroup esr
psu addto ugroup esr esr:esr@osm
psu create ugroup ilc
psu addto ugroup ilc ilc:ilc@osm
psu create ugroup t2k
psu addto ugroup t2k t2k:t2k@osm
psu create ugroup dteam
psu addto ugroup dteam dteam:dteam@osm
psu create ugroup any-store
psu addto ugroup any-store *@*
#
# The pools ...
#
psu create pool csfnfs60_1
psu create pool csfnfs61_2
psu create pool csfnfs54_2
psu create pool ilc_1
psu create pool csfnfs60_2
psu create pool csfnfs61_3
psu create pool csfnfs62_2
psu create pool csfnfs61_4
psu create pool lhcb_61_1
psu create pool csfnfs63_1
psu create pool cms_35
psu create pool zeus_1
psu create pool cms_62_2
psu create pool pheno_1
psu create pool csfnfs54_4
psu create pool csfnfs51_3
psu create pool cms_62_1
psu create pool cms_63_1
psu create pool atlas_60_2
psu create pool csfnfs39_2
psu create pool csfnfs63_4
psu create pool csfnfs60_4
psu create pool csfnfs51_4
psu create pool magic_1
psu create pool csfnfs63_3
psu create pool cms_36
psu create pool cms_34
psu create pool cms_63_2
psu create pool csfnfs60_3
psu create pool h1_2
psu create pool csfnfs61_1
psu create pool csfnfs54_1
psu create pool atlas_60_1
psu create pool lhcb_61_2
psu create pool csfnfs39_1
psu create pool csfnfs63_2
psu create pool csfnfs62_1
psu create pool cms_37
psu create pool csfnfs54_3
psu create pool csfnfs42_1
psu create pool csfnfs51_1
psu create pool t2k_1
psu create pool bio_1
psu create pool csfnfs62_3
psu create pool csfnfs42_2
psu create pool csfnfs51_2
psu create pool babar_1
psu create pool esr_1
psu create pool csfnfs62_4
psu create pool h1_1
#
# The pool groups ...
#
psu create pgroup esr-pgroup
psu addto pgroup esr-pgroup esr_1
psu create pgroup ilc-pgroup
psu addto pgroup ilc-pgroup ilc_1
psu create pgroup zeus-pgroup
psu addto pgroup zeus-pgroup zeus_1
psu create pgroup bio-pgroup
psu addto pgroup bio-pgroup bio_1
psu create pgroup lhcb-tape-pgroup
psu addto pgroup lhcb-tape-pgroup csfnfs60_1
psu addto pgroup lhcb-tape-pgroup csfnfs62_1
psu addto pgroup lhcb-tape-pgroup csfnfs63_1
psu addto pgroup lhcb-tape-pgroup csfnfs61_1
psu create pgroup dteam-pgroup
psu addto pgroup dteam-pgroup csfnfs51_4
psu addto pgroup dteam-pgroup csfnfs61_2
psu addto pgroup dteam-pgroup csfnfs54_2
psu addto pgroup dteam-pgroup csfnfs61_3
psu addto pgroup dteam-pgroup csfnfs60_2
psu addto pgroup dteam-pgroup csfnfs63_3
psu addto pgroup dteam-pgroup csfnfs62_2
psu addto pgroup dteam-pgroup cms_36
psu addto pgroup dteam-pgroup csfnfs61_4
psu addto pgroup dteam-pgroup cms_34
psu addto pgroup dteam-pgroup cms_63_2
psu addto pgroup dteam-pgroup csfnfs60_3
psu addto pgroup dteam-pgroup lhcb_61_1
psu addto pgroup dteam-pgroup cms_35
psu addto pgroup dteam-pgroup csfnfs54_1
psu addto pgroup dteam-pgroup atlas_60_1
psu addto pgroup dteam-pgroup cms_62_2
psu addto pgroup dteam-pgroup csfnfs39_1
psu addto pgroup dteam-pgroup lhcb_61_2
psu addto pgroup dteam-pgroup csfnfs54_4
psu addto pgroup dteam-pgroup csfnfs63_2
psu addto pgroup dteam-pgroup csfnfs51_3
psu addto pgroup dteam-pgroup cms_37
psu addto pgroup dteam-pgroup cms_62_1
psu addto pgroup dteam-pgroup cms_63_1
psu addto pgroup dteam-pgroup csfnfs54_3
psu addto pgroup dteam-pgroup csfnfs42_1
psu addto pgroup dteam-pgroup csfnfs51_1
psu addto pgroup dteam-pgroup atlas_60_2
psu addto pgroup dteam-pgroup csfnfs62_3
psu addto pgroup dteam-pgroup csfnfs42_2
psu addto pgroup dteam-pgroup csfnfs39_2
psu addto pgroup dteam-pgroup csfnfs51_2
psu addto pgroup dteam-pgroup csfnfs63_4
psu addto pgroup dteam-pgroup csfnfs62_4
psu addto pgroup dteam-pgroup csfnfs60_4
psu create pgroup babar-pgroup
psu addto pgroup babar-pgroup babar_1
psu create pgroup default
psu addto pgroup default csfnfs60_1
psu addto pgroup default csfnfs61_2
psu addto pgroup default csfnfs54_2
psu addto pgroup default ilc_1
psu addto pgroup default csfnfs60_2
psu addto pgroup default csfnfs61_3
psu addto pgroup default csfnfs62_2
psu addto pgroup default csfnfs61_4
psu addto pgroup default lhcb_61_1
psu addto pgroup default csfnfs63_1
psu addto pgroup default cms_35
psu addto pgroup default zeus_1
psu addto pgroup default cms_62_2
psu addto pgroup default pheno_1
psu addto pgroup default csfnfs54_4
psu addto pgroup default csfnfs51_3
psu addto pgroup default cms_62_1
psu addto pgroup default cms_63_1
psu addto pgroup default atlas_60_2
psu addto pgroup default csfnfs39_2
psu addto pgroup default csfnfs63_4
psu addto pgroup default csfnfs60_4
psu addto pgroup default csfnfs51_4
psu addto pgroup default magic_1
psu addto pgroup default csfnfs63_3
psu addto pgroup default cms_36
psu addto pgroup default cms_34
psu addto pgroup default cms_63_2
psu addto pgroup default csfnfs60_3
psu addto pgroup default h1_2
psu addto pgroup default csfnfs61_1
psu addto pgroup default csfnfs54_1
psu addto pgroup default atlas_60_1
psu addto pgroup default lhcb_61_2
psu addto pgroup default csfnfs39_1
psu addto pgroup default csfnfs63_2
psu addto pgroup default csfnfs62_1
psu addto pgroup default cms_37
psu addto pgroup default csfnfs54_3
psu addto pgroup default csfnfs42_1
psu addto pgroup default csfnfs51_1
psu addto pgroup default t2k_1
psu addto pgroup default bio_1
psu addto pgroup default csfnfs62_3
psu addto pgroup default csfnfs42_2
psu addto pgroup default csfnfs51_2
psu addto pgroup default babar_1
psu addto pgroup default esr_1
psu addto pgroup default csfnfs62_4
psu addto pgroup default h1_1
psu create pgroup pheno-pgroup
psu addto pgroup pheno-pgroup pheno_1
psu create pgroup cms-tape-pgroup
psu addto pgroup cms-tape-pgroup csfnfs60_1
psu addto pgroup cms-tape-pgroup csfnfs62_1
psu addto pgroup cms-tape-pgroup csfnfs63_1
psu addto pgroup cms-tape-pgroup csfnfs61_1
psu create pgroup atlas-tape-pgroup
psu addto pgroup atlas-tape-pgroup csfnfs60_1
psu addto pgroup atlas-tape-pgroup csfnfs62_1
psu addto pgroup atlas-tape-pgroup csfnfs63_1
psu addto pgroup atlas-tape-pgroup csfnfs61_1
psu create pgroup lhcb-pgroup
psu addto pgroup lhcb-pgroup csfnfs61_2
psu addto pgroup lhcb-pgroup csfnfs39_2
psu addto pgroup lhcb-pgroup lhcb_61_1
psu addto pgroup lhcb-pgroup lhcb_61_2
psu addto pgroup lhcb-pgroup csfnfs39_1
psu create pgroup readonly
psu create pgroup magic-pgroup
psu addto pgroup magic-pgroup magic_1
psu create pgroup t2k-pgroup
psu addto pgroup t2k-pgroup t2k_1
psu create pgroup cms-pgroup
psu addto pgroup cms-pgroup csfnfs63_2
psu addto pgroup cms-pgroup csfnfs51_4
psu addto pgroup cms-pgroup csfnfs51_3
psu addto pgroup cms-pgroup cms_37
psu addto pgroup cms-pgroup cms_62_1
psu addto pgroup cms-pgroup cms_63_1
psu addto pgroup cms-pgroup csfnfs42_1
psu addto pgroup cms-pgroup csfnfs51_1
psu addto pgroup cms-pgroup csfnfs62_2
psu addto pgroup cms-pgroup cms_36
psu addto pgroup cms-pgroup csfnfs42_2
psu addto pgroup cms-pgroup cms_34
psu addto pgroup cms-pgroup cms_63_2
psu addto pgroup cms-pgroup csfnfs51_2
psu addto pgroup cms-pgroup cms_35
psu addto pgroup cms-pgroup cms_62_2
psu create pgroup hone-pgroup
psu addto pgroup hone-pgroup h1_2
psu addto pgroup hone-pgroup h1_1
psu create pgroup atlas-pgroup
psu addto pgroup atlas-pgroup csfnfs54_4
psu addto pgroup atlas-pgroup csfnfs54_2
psu addto pgroup atlas-pgroup csfnfs54_1
psu addto pgroup atlas-pgroup csfnfs60_2
psu addto pgroup atlas-pgroup atlas_60_1
psu addto pgroup atlas-pgroup csfnfs54_3
psu addto pgroup atlas-pgroup atlas_60_2
psu create pgroup dteam-tape-pgroup
psu addto pgroup dteam-tape-pgroup csfnfs60_1
psu addto pgroup dteam-tape-pgroup csfnfs62_1
psu addto pgroup dteam-tape-pgroup csfnfs63_1
psu addto pgroup dteam-tape-pgroup csfnfs61_1
#
# The links ...
#
psu create link bio-link bio
psu set link bio-link -readpref=10 -writepref=10 -cachepref=10
psu add link bio-link bio-pgroup
psu create link ilc-link ilc
psu set link ilc-link -readpref=10 -writepref=10 -cachepref=10
psu add link ilc-link ilc-pgroup
psu create link lhcb-link lhcb
psu set link lhcb-link -readpref=10 -writepref=10 -cachepref=10
psu add link lhcb-link lhcb-pgroup
psu create link zeus-link zeus
psu set link zeus-link -readpref=10 -writepref=10 -cachepref=10
psu add link zeus-link zeus-pgroup
psu create link lhcb-tape-link lhcb-tape world-net
psu set link lhcb-tape-link -readpref=10 -writepref=10 -cachepref=10
psu add link lhcb-tape-link lhcb-tape-pgroup
psu create link hone-link hone
psu set link hone-link -readpref=10 -writepref=10 -cachepref=10
psu add link hone-link hone-pgroup
psu create link magic-link magic
psu set link magic-link -readpref=0 -writepref=0 -cachepref=0
psu add link magic-link magic-pgroup
psu create link t2k-link t2k
psu set link t2k-link -readpref=10 -writepref=10 -cachepref=10
psu add link t2k-link t2k-pgroup
psu create link babar-link babar
psu set link babar-link -readpref=10 -writepref=10 -cachepref=10
psu add link babar-link babar-pgroup
psu create link dteam-tape-link dteam-tape
psu set link dteam-tape-link -readpref=10 -writepref=10 -cachepref=10
psu add link dteam-tape-link dteam-tape-pgroup
psu create link read-only-link world-net
psu set link read-only-link -readpref=10 -writepref=0 -cachepref=0
psu add link read-only-link readonly
psu create link esr-link esr
psu set link esr-link -readpref=10 -writepref=10 -cachepref=10
psu add link esr-link esr-pgroup
psu create link cms-tape-link cms-tape world-net
psu set link cms-tape-link -readpref=10 -writepref=10 -cachepref=10
psu add link cms-tape-link cms-tape-pgroup
psu create link cms-link cms world-net
psu set link cms-link -readpref=10 -writepref=10 -cachepref=10
psu add link cms-link cms-pgroup
psu create link atlas-tape-link atlas-tape world-net
psu set link atlas-tape-link -readpref=10 -writepref=10 -cachepref=10
psu add link atlas-tape-link atlas-tape-pgroup
psu create link pheno-link pheno
psu set link pheno-link -readpref=10 -writepref=10 -cachepref=10
psu add link pheno-link pheno-pgroup
psu create link atlas-link atlas world-net
psu set link atlas-link -readpref=10 -writepref=10 -cachepref=10
psu add link atlas-link atlas-pgroup
psu create link dteam-link dteam world-net
psu set link dteam-link -readpref=10 -writepref=10 -cachepref=10
psu add link dteam-link dteam-pgroup
#
# Submodule [rc] : class diskCacheV111.poolManager.RequestContainerV5
#
rc onerror suspend
rc set max retries 3
rc set retry 900
rc set warning path billing
rc set poolpingtimer 600
rc set slope 0.0
rc set p2p on
rc set p2p oncost
rc set stage oncost off
rc set stage on
set costcuts -idle=0.0 -p2p=2.0 -alert=0.0 -halt=0.0 -fallback=0.0
rc set max copies 500
rc set max restore unlimited
rc set sameHostCopy besteffort
rc set max threads 0