John, I have attached what I use for testing upgrades of the CIP (or
rather the BDII which sits in front of the CIP.)
Basically, you do
ldapsearch -x -LLL -H ldap://${OLDBDIIHOST}:2170/ -b ... \
>oldbdii.tmp
into a file, for each of the BDIIs.
Then
ldifsort.pl -k dn oldbdii.tmp >oldbdii.ldif
and similarly s/old/new/, mutatis mutandis.
Finally
ldifdiff.pl -k dn newbdii.ldif oldbdii.ldif
This'll give you the delta (in LDAP change format), to go from the old
and the new. Note that it doesn't then tell you what the _old_ value
was. Also it assumes relatively consistent naming of things (ie DNs).
Sometimes I have masked out numbers (dynamic stuff) by sed'ing numbers
into XXXXXX or something, because numbers generally change from one
snapshot to the next - of course numbers, too, need checking...
Also, don't forget the Flavian test (somewhat customised by me) - you
fail this at your peril (as I know only too well).
#!/bin/ksh
for VO in atlas cms lhcb dteam alice minos mice hone ilc t2k superb na62
snoplus
do echo "***${VO}***"
ldapsearch -LLL -x -H ldap://srmbdii02.gridpp.rl.ac.uk:2170/ -b o=grid \
"(|(&(GlueChunkKey=GlueSEUniqueID=srm-${VO}.gridpp.rl.ac.uk)(|(GlueSAAccessControlBaseRule=ops)(GlueSAAccessControlBaseRule=VO:ops)))(&(GlueChunkKey=GlueSEUniqueID=srm-${VO}.gridpp.rl.ac.uk)(|(GlueVOInfoAccessControlBaseRule=ops)(GlueVOInfoAccessControlBaseRule=VO:ops)))(&(GlueServiceUniqueID=*://srm-${VO}.gridpp.rl.ac.uk*)(GlueServiceVersion=2.*)(GlueServiceType=srm*)))"
\
GlueServiceEndpoint GlueSAPath GlueVOInfoPath
done
Note that the loop isn't really once per VO, it's one per SRM endpoint
(or actually GlueSEUniqueID, but they are required to be SRM endpoint
hostnames).
Learn to identify healthy output - there should be three, four with a
space token for the VO. As a rule.
Enjoy
--jens
On 06/09/2012 15:09, Chris Brew wrote:
> Hi John,
>
> You're probably a bit beyond this point but I'm still proud of the script-fu
> that generated this for comparing the output of two site bdiis:
>
> comm -3 <(ldapsearch -x -H ldap://<OLD_BDII_NAME>:2170 -b
> mds-vo-name=<SITE_NAME>,o=grid | sed -ne '1{h;n;};/^ /!{x;p;};/^ /{H;g;s/\n
> //;h;};${g;s/\n //;p;}' | grep '^dn: ' | sort) <(ldapsearch -x -H
> ldap://<NEW_BDII_NAME>:2170 -b mds-vo-name=<SITE_NAME>,o=grid | sed -ne
> '1{h;n;};/^ /!{x;p;};/^ /{H;g;s/\n //;h;};${g;s/\n //;p;}' | grep '^dn: ' |
> sort)
>
> Yours,
> Chris (who says it's only perl that looks like line noise) Brew.
>
>> -----Original Message-----
>> From: Testbed Support for GridPP member institutes [mailto:TB-
>> [log in to unmask]] On Behalf Of John Hill
>> Sent: 06 September 2012 15:01
>> To: [log in to unmask]
>> Subject: Site BDII move
>>
>> This morning I started the move to a new Site BDII at Cambridge (SL6,
>> EMI-2). MyEGI and NAGIOS have both noticed the change of machine, but
>> GSTAT hasn't as yet. Is this just a matter of being patient? Is there
>> anywhere else I should be checking to ensure that the new Site BDII is
>> recognised and working correctly before I shut down the old one?
>> Thanks,
>> John
--
Scanned by iCritical.
#! /usr/bin/perl
# $Id: ldifsort.pl,v 1.9 2005/04/03 20:20:24 subbarao Exp $
=head1 NAME
ldifsort.pl - Sorts an LDIF file by the specified key attribute. The sorted
version is written to standard output.
=head1 DESCRIPTION
Sorts an LDIF file by the specified key attribute.
=head1 SYNOPSIS
ldifsort.pl B<-k keyattr> [B<-andc>] file.ldif
=over 4
=item B<-k>
Specifies the key attribute for making sort comparisons. If 'dn' is
specified, sorting is done by the full DN string, which can be composed of
different attributes for different entries.
=item B<-a>
Specifies that attributes within a given entry should also be sorted. This
has the side effect of removing all comments and line continuations in the
LDIF file.
=item B<-n>
Specifies numeric comparisons on the key attribute. Otherwise string
comparisons are done.
=item B<-d>
Specifies that the key attribute is a DN. Comparisons are done on a
DN-normalized version of attribute values. This is the default
behavior if 'dn' is passed as the argument to B<-k>.
=item B<-c>
Specifies case-insensitive comparisons on the key attribute. This is the
default behavior if 'dn' is passed as the argument to B<-k>.
=back
=head1 AUTHOR
Kartik Subbarao E<lt>[log in to unmask]<gt>
=cut
use Net::LDAP::Util qw(canonical_dn);
use MIME::Base64;
use Getopt::Std;
use strict;
my %args;
getopts("k:andc", \%args);
my $keyattr = $args{k};
my $sortattrs = $args{a};
my $ciscmp = $args{c};
my $ldiffile = $ARGV[0];
die "usage: $0 -k keyattr [-n] [-d] [-c] ldiffile\n"
unless $keyattr && $ldiffile;
$/ = "";
open(LDIFH, $ldiffile) || die "$ldiffile: $!\n";
my $pos = 0;
my @valuepos;
while (<LDIFH>) {
my $value;
1 while s/^($keyattr:.*)?\n /$1/im; # Handle line continuations
if (/^$keyattr(::?) (.*)$/im) {
$value = $2;
$value = decode_base64($value) if $1 eq '::';
}
$value = lc($value) if $ciscmp;
push @valuepos, [ $value, $pos ];
$pos = tell;
}
sub cmpattr { $a->[0] cmp $b->[0] }
sub cmpattrnum { $a->[0] <=> $b->[0] }
my %canonicaldns;
sub cmpdn {
my $cadn = ($canonicaldns{$a->[0]} ||= lc(canonical_dn($a->[0])));
my $cbdn = ($canonicaldns{$b->[0]} ||= lc(canonical_dn($b->[0])));
$cadn cmp $cbdn;
}
my $cmpfunc;
if ($args{d} || lc($keyattr) eq 'dn') { $cmpfunc = \&cmpdn }
elsif ($args{n}) { $cmpfunc = \&cmpattrnum }
else { $cmpfunc = \&cmpattr; }
my @sorted;
@sorted = sort $cmpfunc @valuepos;
foreach my $valuepos (@sorted) {
seek(LDIFH, $valuepos->[1], 0);
my $entry = <LDIFH>;
if ($sortattrs) {
$entry =~ s/\n //mg; # collapse line continuations
my @lines = grep(!/^#/, split(/\n/, $entry));
my $dn = shift(@lines);
print "$dn\n", join("\n", sort @lines), "\n\n";
}
else {
print $entry;
print "\n" if $entry !~ /\n\n$/;
}
}
#! /usr/bin/perl
# $Id: ldifdiff.pl,v 3.7 2005/03/15 14:22:45 subbarao Exp $
=head1 NAME
ldifdiff.pl -- Generates LDIF change diff between two sorted LDIF files.
=head1 DESCRIPTION
ldifdiff.pl takes as input two sorted LDIF files, source and target, and
generates on standard output the LDIF changes needed to transform the target
into the source.
=head1 SYNOPSIS
ldifdiff.pl B<-k|--keyattr keyattr> [B<-a|--sourceattrs attr1,attr2,...>] [B<-c|--ciscmp attr1,...>] [B<--dnattrs attr1,...>] [B<--sharedattrs attr1,...>] B<sourcefile> B<targetfile>
=head1 OPTIONS
=over 4
=item B<-k|--keyattr> keyattr
Specifies the key attribute to use when comparing source and target entries.
Entries in both LDIF files must be sorted by this attribute for comparisons to
be meaningful. F<ldifsort.pl> can be used to sort LDIF files by a given
attribute.
=item B<-a|--sourceattrs attr1,attr2,...>
(Optional) Specifies a list of attributes to consider when comparing
source and target entries. By default, all attributes are considered.
=item B<-c|--ciscmp attr1,...>
(Optional) Compare values of the specified attributes case-insensitively. By
default, comparisons are case-sensitive.
=item B<--dnattrs attr1,...>
(Optional) Specifies a list of attributes to be treated as DNs when being
compared. The default set is manager, member, owner and uniqueMember.
=item B<--sharedattrs attr1,...>
(Optional) Specifies a list of attribues to be treated as "shared" attributes,
where the source may not be a sole authoritative source. When modifying
these attributes, separate "delete" and "add" LDIF changes are generated,
instead of a single "replace" change. The default set is objectClass.
=item B<sourcefile>
Specifies the source LDIF file.
=item B<targetfile>
Specifies the target LDIF file.
=back
=cut
use Net::LDAP;
use Net::LDAP::LDIF;
use Net::LDAP::Util qw(canonical_dn);
use Getopt::Long;
use strict;
my @sourceattrs;
my (%ciscmp, %dnattrs, %sharedattrs);
my $keyattr;
GetOptions('a|sourceattrs=s' => sub { @sourceattrs = split(/,/, $_[1]) },
'c|ciscmp=s' => sub { my @a = split(/,/,lc $_[1]); @ciscmp{@a} = (1) x @a },
'dnattrs=s' => sub { my @a = split(/,/,lc $_[1]); @dnattrs{@a} = (1) x @a },
'k|keyattr=s' => \$keyattr,
'sharedattrs=s' => sub {my @a=split(/,/,lc $_[1]);@sharedattrs{@a}=(1) x @a}
);
%ciscmp = (objectclass => 1, manager => 1, member => 1, owner => 1,
uniquemember => 1)
unless keys %ciscmp;
%dnattrs = (manager => 1, member => 1, owner => 1, uniquemember => 1)
unless keys %dnattrs;
%sharedattrs = (objectclass => 1)
unless keys %sharedattrs;
my ($sourcefile, $targetfile);
$sourcefile = shift; $targetfile = shift;
die "usage: $0 -k|--keyattr keyattr [-a|--sourceattrs attr1,attr2,...] [-c|--ciscmp attr1,...] [--dnattrs attr1,...] [--sharedattrs attr1,...] sourcefile targetfile\n"
unless $keyattr && $sourcefile && $targetfile;
my $source = Net::LDAP::LDIF->new($sourcefile)
or die "Can't open LDIF file $sourcefile: $!\n";
my $target = Net::LDAP::LDIF->new($targetfile)
or die "Can't open LDIF file $targetfile: $!\n";
my $ldifout = Net::LDAP::LDIF->new('-', 'w');
$ldifout->{change} = 1;
$ldifout->{wrap} = 78;
diff($source, $target);
exit;
# Gets the relative distinguished name (RDN) attribute
sub rdnattr { ($_[0]->dn =~ /^(.*?)=/)[0] }
# Gets the relative distinguished name (RDN) value
sub rdnval { my $rv = ($_[0]->dn =~ /=(.*)/)[0]; $rv =~ s/(?<!\\),.*//; $rv }
sub cmpEntries
{
my ($a, $b) = @_;
if ($keyattr =~ /^dn$/i) {
return lc(canonical_dn($a->dn)) cmp lc(canonical_dn($b->dn))
}
else {
my $aval = $a->get_value($keyattr);
my $bval = $b->get_value($keyattr);
if ($ciscmp{$keyattr}) {
$aval = lc($aval);
$bval = lc($bval);
}
return $aval cmp $bval;
}
}
# Diffs two LDIF data sources
sub diff
{
my ($source, $target) = @_;
my ($sourceentry, $targetentry, $incr_source, $incr_target, @ldifchanges);
$sourceentry = $source->read_entry();
$targetentry = $target->read_entry();
while () {
# End of all data
last if !$sourceentry && !$targetentry;
# End of source data, but more target data. Delete.
if (!$sourceentry && $targetentry) {
$targetentry->delete;
$ldifout->write_entry($targetentry);
$incr_target = 1, next;
}
# End of target data, but more data in source. Add.
if ($sourceentry && !$targetentry) {
$ldifout->write_entry($sourceentry);
$incr_source = 1, next;
}
# Check if the current source entry has a higher sort position than
# the current target. If so, we interpret this to mean that the
# target entry no longer exists on the source. Issue a delete to LDAP.
if (cmpEntries($sourceentry, $targetentry) > 0) {
$targetentry->delete;
$ldifout->write_entry($targetentry);
$incr_target = 1, next;
}
# Check if the current target entry has a higher sort position than
# the current source entry. If so, we interpret this to mean that the
# source entry doesn't exist on the target. Issue an add to LDAP.
if (cmpEntries($targetentry, $sourceentry) > 0) {
$ldifout->write_entry($sourceentry);
$incr_source = 1, next;
}
# When we get here, we're dealing with the same person in $sourceentry
# and $targetentry. Compare the data and generate the update.
# If a modRDN is necessary, it needs to happen before other mods
if (lc(canonical_dn($sourceentry->dn))
ne lc(canonical_dn($targetentry->dn))) {
my $rdnattr = rdnattr($sourceentry);
my $rdnval = rdnval($sourceentry);
$targetentry->{changetype} = 'modrdn';
$targetentry->add(newrdn => "$rdnattr=$rdnval",
deleteoldrdn => '1');
$ldifout->write_entry($targetentry);
$targetentry->delete('newrdn');
$targetentry->delete('deleteoldrdn');
delete($targetentry->{changetype});
$targetentry->dn($sourceentry->dn);
$targetentry->replace($rdnattr, $sourceentry->get_value($rdnattr))
if $sourceentry->exists($rdnattr);
}
# Check for differences and generate LDIF as appropriate
updateFromEntry($sourceentry, $targetentry, @sourceattrs);
$ldifout->write_entry($targetentry) if @{$targetentry->{changes}};
$incr_source = 1, $incr_target = 1, next;
} continue {
if ($incr_source) {
$sourceentry = $source->read_entry(); $incr_source = 0;
}
if ($incr_target) {
$targetentry = $target->read_entry(); $incr_target = 0;
}
}
}
# Generate LDIF to update $target with information in $source.
# Optionally restrict the set of attributes to consider.
sub updateFromEntry
{
my ($source, $target, @attrs) = @_;
my ($attr, $val, $ldifstr);
unless (@attrs) {
# add all source entry attributes
@attrs = $source->attributes;
# add any other attributes we haven't seen from the target entry
foreach my $tattr ($target->attributes) {
push(@attrs, $tattr) unless grep(/^$tattr$/i, @attrs)
}
}
$target->{changetype} = 'modify';
foreach $attr (@attrs) {
my $lcattr = lc $attr;
next if $lcattr eq 'dn'; # Can't handle modrdn here
# Build lists of unique values in the source and target, to
# speed up comparisons.
my @sourcevals = $source->get_value($attr);
my @targetvals = $target->get_value($attr);
my (%sourceuniqvals, %targetuniqvals);
foreach (@sourcevals) {
my ($origval, $val) = ($_, $_);
$val = lc $val if $ciscmp{$lcattr};
# Get rid of spaces after non-escaped commas in DN attrs
$val =~ s/(?<!\\),\s+/,/g if $dnattrs{$lcattr};
$sourceuniqvals{$val} = $origval;
}
foreach (@targetvals) {
my ($origval, $val) = ($_, $_);
$val = lc $val if $ciscmp{$lcattr};
# Get rid of spaces after non-escaped commas in DN attrs
$val =~ s/(?<!\\),\s+/,/g if $dnattrs{$lcattr};
$targetuniqvals{$val} = $origval;
}
foreach my $val (keys %sourceuniqvals) {
if (exists $targetuniqvals{$val}) {
delete $sourceuniqvals{$val};
delete $targetuniqvals{$val};
}
}
# Move on if there are no differences
next unless keys(%sourceuniqvals) || keys(%targetuniqvals);
# Make changes as appropriate
if ($sharedattrs{$lcattr}) {
# For 'shared' attributes (e.g. objectclass) where $source may not
# be a sole authoritative source, we issue separate delete and
# add modifications instead of a single replace.
$target->delete($attr => [ values(%targetuniqvals) ])
if keys(%targetuniqvals);
$target->add($attr => [ values(%sourceuniqvals) ])
if keys(%sourceuniqvals);
}
else {
# Issue a replace or delete as needed
if (@sourcevals) { $target->replace($attr => [ @sourcevals ]) }
else { $target->delete($attr) }
}
}
# Get rid of the "changetype: modify" if there were no changes
delete($target->{changetype}) unless @{$target->{changes}};
}
=back
=head1 AUTHOR
Kartik Subbarao E<lt>[log in to unmask]<gt>
=cut
|
