Hi Kashif,
On 10/03/2013 03:20 PM, Kashif Mohammad wrote:
> As I am moving both of our production CE and torque server together so my plan is to completely drain torque server and then publish everything one final time. After that I can move to emi3 publisher. In that case probably I don't need any db migration. Is it right ?
>
>
My guess - yes. But I'd ask Stuart Pullinger (cc'd). He indicated that
migrate_python.py or update process may be too weak for an "in-flight"
update, so to speak.
Cheers,
Steve
On 10/03/2013 03:20 PM, Kashif Mohammad wrote:
> Hi Steve
>
>>> We are planning to move to SL6 EMI3 CREAMCE and WN next week. I was
>> thinking of upgrading to emi3 apel publisher but I got the impression
>> that it has still some issues. Is there any site in UK which
>> successfully upgraded to EMI3 APEL publisher ?
>>> Thanks
>>> Kashif
>> We are successfully publishing current data with APEL3.
>>
>> But the problem came (and persist) with the upgrade plan referred to
>> from here:
>> https://twiki.cern.ch/twiki/bin/view/EMI/EMI3APELClient
>>
>> i.e.
>> https://twiki.cern.ch/twiki/pub/EMI/EMI3APELClient/APEL_Client_Upgrade_
>> Plan.pdf
>>
>> The upgrade plan advises us to extract from the old database a month
>> or
>> two of
>> whole month records, and fit them in the new database schema. This (is
>> supposed to)
>> enable a clean transfer.
> As I am moving both of our production CE and torque server together so my plan is to completely drain torque server and then publish everything one final time. After that I can move to emi3 publisher. In that case probably I don't need any db migration. Is it right ?
>
> Cheers
> Kashif
>
>
>
>> Unfortunately, the import routine (called migrate_apel.pl) is not
>> universally reliable for reasons TBD.
>> Net result: system thinks we have transmitted too few records, and asks
>> for GAP publisher
>> for historic (overlap) months. Current months seem to work.
>>
>> Bug: https://ggus.eu/ws/ticket_info.php?ticket=97528
>>
>> To install, use puppet to put on packages, then script to finish, see
>> below.
>> It has to be done big-bang style.
>>
>> Cheers,
>>
>> Steve
>>
>>
>> :-----------
>>
>> * Script to finish on client, e.g. cream
>>
>> root@hepgrid5 scripts]# cat config_apel3_parser.sh.clean
>> #!/bin/bash
>> # --------------------------------------------------------
>> # Name: config_apel_parser.sh
>> # Purpose: Configure the APEL parser
>> # Author: Steve Jones <[log in to unmask]>
>> # Date: 18 Sept 2013
>> # --------------------------------------------------------
>>
>> # Secure config files
>> chmod 600 /etc/apel/parser.cfg
>>
>> # Do the easy changes
>>
>> perl -pi -e 's/^name = .*/name = apelclient/g' /etc/apel/parser.cfg
>> perl -pi -e 's/^hostname = .*/hostname = hepgrid95.ph.liv.ac.uk/g'
>> /etc/apel/parser.cfg
>> perl -pi -e 's/^username = .*/username =
>> apelclient/g' /etc/apel/parser.cfg
>> perl -pi -e 's/^password = .*/password = somepass/g'
>> /etc/apel/parser.cfg
>> perl -pi -e 's/^site_name = .*/site_name = UKI-NORTHGRID-LIV-HEP/g'
>> /etc/apel/parser.cfg
>> perl -pi -e 's/^lrms_server = .*/lrms_server =
>> hepgrid96.ph.liv.ac.uk/g'
>> /etc/apel/parser.cfg
>> perl -pi -e 's/^lrms_server = .*/lrms_server =
>> hepgrid96.ph.liv.ac.uk/g'
>> /etc/apel/parser.cfg
>> perl -pi -e 's/^type = .*/type = PBS/g' /etc/apel/parser.cfg
>>
>> cp /etc/apel/parser.cfg /etc/apel/parser.cfg.bak
>>
>> # Now do those pesky dirs
>> perl << "XXXX"
>> use strict;
>> my @newLines;
>> my $inBlah = 0; my $inBatch = 0;
>> open(PARSER_CFG,"/etc/apel/parser.cfg") or die("No open
>> /etc/apel/parser.cfg\n");
>> while(<PARSER_CFG>) {
>> my $l = $_;
>> chomp($l);
>> if ($l =~ /^\[/) { $inBlah = 0; $inBatch = 0;}
>> if ($l =~ /^\[blah\]/) { $inBlah = 1; }
>> if ($l =~ /^\[batch\]/) { $inBatch = 1; }
>> if ($l =~ /^dir =/) {
>> if ($inBlah) {
>> $l =~ s/dir \=.*/dir = \/var\/log\/cream\/accounting\//;
>> }
>> if ($inBatch) {
>> $l =~ s/dir \=.*/dir = \/var\/log\/torque_accounting\//;
>> }
>> }
>> push(@newLines,$l);
>> }
>> close(PARSER_CFG);
>> open(PARSER_CFG,">/etc/apel/parser.cfg") or die("No open
>> /etc/apel/parser.cfg\n");
>> foreach my $l (@newLines) {
>> print (PARSER_CFG "$l\n");
>> }
>> close(PARSER_CFG);
>> XXXX
>>
>> * Script to finish on apelserver
>>
>> #!/bin/bash
>> # --------------------------------------------------------
>> # Name: config_apel.sh
>> # Purpose: Configure the APEL publisher
>> # Author: Steve Jones <[log in to unmask]>
>> # Date: 17 Sept 2013
>> # --------------------------------------------------------
>>
>> # Secure config files
>> chmod 600 /etc/apel/client.cfg
>> chmod 600 /etc/apel/sender.cfg
>>
>> # Protect mysql
>> /usr/bin/mysqladmin -u root -h localhost password somepass
>> /usr/bin/mysqladmin -u root -h hepgrid95.ph.liv.ac.uk password
>> somepass
>>
>> # Make the apelclient db
>> /usr/bin/mysql --pass=somepass --exec "create database apelclient"
>>
>> # Let apel parsers commit data to this database
>> /usr/bin/mysql --pass=somepass --exec "grant all on apelclient.* to
>> [log in to unmask] identified by 'somepass'"
>> /usr/bin/mysql --pass=somepass --exec "grant all on apelclient.* to
>> [log in to unmask] identified by 'somepass'"
>> /usr/bin/mysql --pass=somepass --exec "grant all on apelclient.* to
>> [log in to unmask] identified by 'somepass'"
>> /usr/bin/mysql --pass=somepass --exec "grant all on apelclient.* to
>> [log in to unmask] identified by 'somepass'"
>> /usr/bin/mysql --pass=somepass --exec "grant all on apelclient.* to
>> [log in to unmask] identified by 'somepass'"
>> /usr/bin/mysql --pass=somepass --exec "grant all on apelclient.* to
>> [log in to unmask] identified by 'somepass'"
>>
>> # Load the database schema
>> /usr/bin/mysql --pass=somepass apelclient < /usr/share/apel/client.sql
>>
>> # Customise the client.cfg config
>> perl -pi -e 's/username = .*/username = root/g' /etc/apel/client.cfg
>> perl -pi -e 's/password = .*/password = somepass/g'
>> /etc/apel/client.cfg
>> perl -pi -e 's/site_name = .*/site_name = UKI-NORTHGRID-LIV-HEP/g'
>> /etc/apel/client.cfg
>>
>> # Customise the sender.cfg config. Use test first; maybe flip later
>> TEST=1
>> if [ $TEST == 1 ]; then
>> perl -pi -e 's/network:.*/network: TEST-NWOB/g'
>> /etc/apel/sender.cfg
>> perl -pi -e 's/use_ssl:.*/use_ssl: false/g' /etc/apel/sender.cfg
>> perl -pi -e 's/destination:.*/destination:
>> \/queue\/global.accounting.cputest.CENTRAL/g' /etc/apel/sender.cfg
>> else
>> perl -pi -e 's/network:.*/network: PROD/g' /etc/apel/sender.cfg
>> perl -pi -e 's/use_ssl:.*/use_ssl: true/g' /etc/apel/sender.cfg
>> perl -pi -e 's/destination:.*/destination:
>> \/queue\/global.accounting.cpu.central/g' /etc/apel/sender.cfg
>> fi
>>
>> # This dir is needed by the sender
>> if [ ! -d /var/spool/apel/outgoing ]; then
>> mkdir /var/spool/apel/outgoing
>> fi
>>
>>
>>
>>
>>
>> --
>> Steve Jones [log in to unmask]
>> System Administrator office: 220
>> High Energy Physics Division tel (int): 42334
>> Oliver Lodge Laboratory tel (ext): +44 (0)151 794 2334
>> University of Liverpool
>> http://www.liv.ac.uk/physics/hep/
>>
>>
--
Steve Jones [log in to unmask]
System Administrator office: 220
High Energy Physics Division tel (int): 42334
Oliver Lodge Laboratory tel (ext): +44 (0)151 794 2334
University of Liverpool http://www.liv.ac.uk/physics/hep/
|