Hi Steve
> > We are planning to move to SL6 EMI3 CREAMCE and WN next week. I was
> thinking of upgrading to emi3 apel publisher but I got the impression
> that it has still some issues. Is there any site in UK which
> successfully upgraded to EMI3 APEL publisher ?
> >
> > Thanks
> > Kashif
> We are successfully publishing current data with APEL3.
>
> But the problem came (and persist) with the upgrade plan referred to
> from here:
> https://twiki.cern.ch/twiki/bin/view/EMI/EMI3APELClient
>
> i.e.
> https://twiki.cern.ch/twiki/pub/EMI/EMI3APELClient/APEL_Client_Upgrade_
> Plan.pdf
>
> The upgrade plan advises us to extract from the old database a month
> or
> two of
> whole month records, and fit them in the new database schema. This (is
> supposed to)
> enable a clean transfer.
As I am moving both of our production CE and torque server together so my plan is to completely drain torque server and then publish everything one final time. After that I can move to emi3 publisher. In that case probably I don't need any db migration. Is it right ?
Cheers
Kashif
> Unfortunately, the import routine (called migrate_apel.pl) is not
> universally reliable for reasons TBD.
> Net result: system thinks we have transmitted too few records, and asks
> for GAP publisher
> for historic (overlap) months. Current months seem to work.
>
> Bug: https://ggus.eu/ws/ticket_info.php?ticket=97528
>
> To install, use puppet to put on packages, then script to finish, see
> below.
> It has to be done big-bang style.
>
> Cheers,
>
> Steve
>
>
> :-----------
>
> * Script to finish on client, e.g. cream
>
> root@hepgrid5 scripts]# cat config_apel3_parser.sh.clean
> #!/bin/bash
> # --------------------------------------------------------
> # Name: config_apel_parser.sh
> # Purpose: Configure the APEL parser
> # Author: Steve Jones <[log in to unmask]>
> # Date: 18 Sept 2013
> # --------------------------------------------------------
>
> # Secure config files
> chmod 600 /etc/apel/parser.cfg
>
> # Do the easy changes
>
> perl -pi -e 's/^name = .*/name = apelclient/g' /etc/apel/parser.cfg
> perl -pi -e 's/^hostname = .*/hostname = hepgrid95.ph.liv.ac.uk/g'
> /etc/apel/parser.cfg
> perl -pi -e 's/^username = .*/username =
> apelclient/g' /etc/apel/parser.cfg
> perl -pi -e 's/^password = .*/password = somepass/g'
> /etc/apel/parser.cfg
> perl -pi -e 's/^site_name = .*/site_name = UKI-NORTHGRID-LIV-HEP/g'
> /etc/apel/parser.cfg
> perl -pi -e 's/^lrms_server = .*/lrms_server =
> hepgrid96.ph.liv.ac.uk/g'
> /etc/apel/parser.cfg
> perl -pi -e 's/^lrms_server = .*/lrms_server =
> hepgrid96.ph.liv.ac.uk/g'
> /etc/apel/parser.cfg
> perl -pi -e 's/^type = .*/type = PBS/g' /etc/apel/parser.cfg
>
> cp /etc/apel/parser.cfg /etc/apel/parser.cfg.bak
>
> # Now do those pesky dirs
> perl << "XXXX"
> use strict;
> my @newLines;
> my $inBlah = 0; my $inBatch = 0;
> open(PARSER_CFG,"/etc/apel/parser.cfg") or die("No open
> /etc/apel/parser.cfg\n");
> while(<PARSER_CFG>) {
> my $l = $_;
> chomp($l);
> if ($l =~ /^\[/) { $inBlah = 0; $inBatch = 0;}
> if ($l =~ /^\[blah\]/) { $inBlah = 1; }
> if ($l =~ /^\[batch\]/) { $inBatch = 1; }
> if ($l =~ /^dir =/) {
> if ($inBlah) {
> $l =~ s/dir \=.*/dir = \/var\/log\/cream\/accounting\//;
> }
> if ($inBatch) {
> $l =~ s/dir \=.*/dir = \/var\/log\/torque_accounting\//;
> }
> }
> push(@newLines,$l);
> }
> close(PARSER_CFG);
> open(PARSER_CFG,">/etc/apel/parser.cfg") or die("No open
> /etc/apel/parser.cfg\n");
> foreach my $l (@newLines) {
> print (PARSER_CFG "$l\n");
> }
> close(PARSER_CFG);
> XXXX
>
> * Script to finish on apelserver
>
> #!/bin/bash
> # --------------------------------------------------------
> # Name: config_apel.sh
> # Purpose: Configure the APEL publisher
> # Author: Steve Jones <[log in to unmask]>
> # Date: 17 Sept 2013
> # --------------------------------------------------------
>
> # Secure config files
> chmod 600 /etc/apel/client.cfg
> chmod 600 /etc/apel/sender.cfg
>
> # Protect mysql
> /usr/bin/mysqladmin -u root -h localhost password somepass
> /usr/bin/mysqladmin -u root -h hepgrid95.ph.liv.ac.uk password
> somepass
>
> # Make the apelclient db
> /usr/bin/mysql --pass=somepass --exec "create database apelclient"
>
> # Let apel parsers commit data to this database
> /usr/bin/mysql --pass=somepass --exec "grant all on apelclient.* to
> [log in to unmask] identified by 'somepass'"
> /usr/bin/mysql --pass=somepass --exec "grant all on apelclient.* to
> [log in to unmask] identified by 'somepass'"
> /usr/bin/mysql --pass=somepass --exec "grant all on apelclient.* to
> [log in to unmask] identified by 'somepass'"
> /usr/bin/mysql --pass=somepass --exec "grant all on apelclient.* to
> [log in to unmask] identified by 'somepass'"
> /usr/bin/mysql --pass=somepass --exec "grant all on apelclient.* to
> [log in to unmask] identified by 'somepass'"
> /usr/bin/mysql --pass=somepass --exec "grant all on apelclient.* to
> [log in to unmask] identified by 'somepass'"
>
> # Load the database schema
> /usr/bin/mysql --pass=somepass apelclient < /usr/share/apel/client.sql
>
> # Customise the client.cfg config
> perl -pi -e 's/username = .*/username = root/g' /etc/apel/client.cfg
> perl -pi -e 's/password = .*/password = somepass/g'
> /etc/apel/client.cfg
> perl -pi -e 's/site_name = .*/site_name = UKI-NORTHGRID-LIV-HEP/g'
> /etc/apel/client.cfg
>
> # Customise the sender.cfg config. Use test first; maybe flip later
> TEST=1
> if [ $TEST == 1 ]; then
> perl -pi -e 's/network:.*/network: TEST-NWOB/g'
> /etc/apel/sender.cfg
> perl -pi -e 's/use_ssl:.*/use_ssl: false/g' /etc/apel/sender.cfg
> perl -pi -e 's/destination:.*/destination:
> \/queue\/global.accounting.cputest.CENTRAL/g' /etc/apel/sender.cfg
> else
> perl -pi -e 's/network:.*/network: PROD/g' /etc/apel/sender.cfg
> perl -pi -e 's/use_ssl:.*/use_ssl: true/g' /etc/apel/sender.cfg
> perl -pi -e 's/destination:.*/destination:
> \/queue\/global.accounting.cpu.central/g' /etc/apel/sender.cfg
> fi
>
> # This dir is needed by the sender
> if [ ! -d /var/spool/apel/outgoing ]; then
> mkdir /var/spool/apel/outgoing
> fi
>
>
>
>
>
> --
> Steve Jones [log in to unmask]
> System Administrator office: 220
> High Energy Physics Division tel (int): 42334
> Oliver Lodge Laboratory tel (ext): +44 (0)151 794 2334
> University of Liverpool
> http://www.liv.ac.uk/physics/hep/
>
>
|