Hi Kashif,
On 10/03/2013 02:01 PM, Kashif Mohammad wrote:
> Hi
>
> We are planning to move to SL6 EMI3 CREAMCE and WN next week. I was thinking of upgrading to emi3 apel publisher but I got the impression that it has still some issues. Is there any site in UK which successfully upgraded to EMI3 APEL publisher ?
>
> Thanks
> Kashif
We are successfully publishing current data with APEL3.
But the problem came (and persist) with the upgrade plan referred to
from here:
https://twiki.cern.ch/twiki/bin/view/EMI/EMI3APELClient
i.e.
https://twiki.cern.ch/twiki/pub/EMI/EMI3APELClient/APEL_Client_Upgrade_Plan.pdf
The upgrade plan advises us to extract from the old database a month or
two of
whole month records, and fit them in the new database schema. This (is
supposed to)
enable a clean transfer.
Unfortunately, the import routine (called migrate_apel.pl) is not
universally reliable for reasons TBD.
Net result: system thinks we have transmitted too few records, and asks
for GAP publisher
for historic (overlap) months. Current months seem to work.
Bug: https://ggus.eu/ws/ticket_info.php?ticket=97528
To install, use puppet to put on packages, then script to finish, see below.
It has to be done big-bang style.
Cheers,
Steve
:-----------
* Script to finish on client, e.g. cream
root@hepgrid5 scripts]# cat config_apel3_parser.sh.clean
#!/bin/bash
# --------------------------------------------------------
# Name: config_apel_parser.sh
# Purpose: Configure the APEL parser
# Author: Steve Jones <[log in to unmask]>
# Date: 18 Sept 2013
# --------------------------------------------------------
# Secure config files
chmod 600 /etc/apel/parser.cfg
# Do the easy changes
perl -pi -e 's/^name = .*/name = apelclient/g' /etc/apel/parser.cfg
perl -pi -e 's/^hostname = .*/hostname = hepgrid95.ph.liv.ac.uk/g'
/etc/apel/parser.cfg
perl -pi -e 's/^username = .*/username =
apelclient/g' /etc/apel/parser.cfg
perl -pi -e 's/^password = .*/password = somepass/g' /etc/apel/parser.cfg
perl -pi -e 's/^site_name = .*/site_name = UKI-NORTHGRID-LIV-HEP/g'
/etc/apel/parser.cfg
perl -pi -e 's/^lrms_server = .*/lrms_server = hepgrid96.ph.liv.ac.uk/g'
/etc/apel/parser.cfg
perl -pi -e 's/^lrms_server = .*/lrms_server = hepgrid96.ph.liv.ac.uk/g'
/etc/apel/parser.cfg
perl -pi -e 's/^type = .*/type = PBS/g' /etc/apel/parser.cfg
cp /etc/apel/parser.cfg /etc/apel/parser.cfg.bak
# Now do those pesky dirs
perl << "XXXX"
use strict;
my @newLines;
my $inBlah = 0; my $inBatch = 0;
open(PARSER_CFG,"/etc/apel/parser.cfg") or die("No open
/etc/apel/parser.cfg\n");
while(<PARSER_CFG>) {
my $l = $_;
chomp($l);
if ($l =~ /^\[/) { $inBlah = 0; $inBatch = 0;}
if ($l =~ /^\[blah\]/) { $inBlah = 1; }
if ($l =~ /^\[batch\]/) { $inBatch = 1; }
if ($l =~ /^dir =/) {
if ($inBlah) {
$l =~ s/dir \=.*/dir = \/var\/log\/cream\/accounting\//;
}
if ($inBatch) {
$l =~ s/dir \=.*/dir = \/var\/log\/torque_accounting\//;
}
}
push(@newLines,$l);
}
close(PARSER_CFG);
open(PARSER_CFG,">/etc/apel/parser.cfg") or die("No open
/etc/apel/parser.cfg\n");
foreach my $l (@newLines) {
print (PARSER_CFG "$l\n");
}
close(PARSER_CFG);
XXXX
* Script to finish on apelserver
#!/bin/bash
# --------------------------------------------------------
# Name: config_apel.sh
# Purpose: Configure the APEL publisher
# Author: Steve Jones <[log in to unmask]>
# Date: 17 Sept 2013
# --------------------------------------------------------
# Secure config files
chmod 600 /etc/apel/client.cfg
chmod 600 /etc/apel/sender.cfg
# Protect mysql
/usr/bin/mysqladmin -u root -h localhost password somepass
/usr/bin/mysqladmin -u root -h hepgrid95.ph.liv.ac.uk password somepass
# Make the apelclient db
/usr/bin/mysql --pass=somepass --exec "create database apelclient"
# Let apel parsers commit data to this database
/usr/bin/mysql --pass=somepass --exec "grant all on apelclient.* to
[log in to unmask] identified by 'somepass'"
/usr/bin/mysql --pass=somepass --exec "grant all on apelclient.* to
[log in to unmask] identified by 'somepass'"
/usr/bin/mysql --pass=somepass --exec "grant all on apelclient.* to
[log in to unmask] identified by 'somepass'"
/usr/bin/mysql --pass=somepass --exec "grant all on apelclient.* to
[log in to unmask] identified by 'somepass'"
/usr/bin/mysql --pass=somepass --exec "grant all on apelclient.* to
[log in to unmask] identified by 'somepass'"
/usr/bin/mysql --pass=somepass --exec "grant all on apelclient.* to
[log in to unmask] identified by 'somepass'"
# Load the database schema
/usr/bin/mysql --pass=somepass apelclient < /usr/share/apel/client.sql
# Customise the client.cfg config
perl -pi -e 's/username = .*/username = root/g' /etc/apel/client.cfg
perl -pi -e 's/password = .*/password = somepass/g' /etc/apel/client.cfg
perl -pi -e 's/site_name = .*/site_name = UKI-NORTHGRID-LIV-HEP/g'
/etc/apel/client.cfg
# Customise the sender.cfg config. Use test first; maybe flip later
TEST=1
if [ $TEST == 1 ]; then
perl -pi -e 's/network:.*/network: TEST-NWOB/g' /etc/apel/sender.cfg
perl -pi -e 's/use_ssl:.*/use_ssl: false/g' /etc/apel/sender.cfg
perl -pi -e 's/destination:.*/destination:
\/queue\/global.accounting.cputest.CENTRAL/g' /etc/apel/sender.cfg
else
perl -pi -e 's/network:.*/network: PROD/g' /etc/apel/sender.cfg
perl -pi -e 's/use_ssl:.*/use_ssl: true/g' /etc/apel/sender.cfg
perl -pi -e 's/destination:.*/destination:
\/queue\/global.accounting.cpu.central/g' /etc/apel/sender.cfg
fi
# This dir is needed by the sender
if [ ! -d /var/spool/apel/outgoing ]; then
mkdir /var/spool/apel/outgoing
fi
--
Steve Jones [log in to unmask]
System Administrator office: 220
High Energy Physics Division tel (int): 42334
Oliver Lodge Laboratory tel (ext): +44 (0)151 794 2334
University of Liverpool http://www.liv.ac.uk/physics/hep/
