Hi,
I can connect to the SE fine (using a gridpp proxy):
lx00:~ :~] xrd lcgse01.phy.bris.ac.uk
(C) 2004-2010 by the Xrootd group. Xrootd version: v4.4.0
Welcome to the xrootd command line interface.
Type 'help' for a list of available commands.
root://lcgse01.phy.bris.ac.uk:1094//> ls
dr-x(019) 0 2015-03-18 13:55:47 /alice
dr-x(019) 0 2015-03-18 13:55:43 /atlas
dr-x(019) 2043990715 2016-09-28 20:39:39 /cms
dr-x(019) 2564256 2016-11-29 09:18:13 /dteam
dr-x(019) 0 2016-06-08 10:31:19 /gridpp
dr-x(019) 0 2015-03-18 13:55:42 /lhcb
dr-x(019) 250 2017-08-30 15:40:00 /ops
dr-x(019) 0 2016-06-08 10:31:20 /vo.southgrid.org
So the gridpp directory is empty,
But when I try and write to it (unless I have a typo) I get:
xrdcp -d3 /bin/bash
root://lcgse01.phy.bris.ac.uk//dpm/phy.bris.ac.uk/home/gridpp/gridpp/user/d/daniela.bauer/upload1.txt
[...]
[2017-08-30 17:01:32.196602 +0100][Dump ][XRootD ]
[lcgse01.phy.bris.ac.uk:1094] Got a kXR_error response to request
kXR_open (file:
/dpm/phy.bris.ac.uk/home/gridpp/gridpp/user/d/daniela.bauer/upload1.txt?oss.asize=942200,
mode: 0644, flags: kXR_new kXR_open_updt kXR_async kXR_retstat )
[3010] Unable to create
/dpm/phy.bris.ac.uk/home/gridpp/gridpp/user/d/daniela.bauer/upload1.txt;
Permission denied
[...]
Regards,
Daniela
On 30 August 2017 at 16:47, Luke Kreczko <[log in to unmask]> wrote:
> Hi Daniela, hi Sam,
>
> DIRAC has now discovered the Bristol storage
> I might run some tests later.
>
> Awesome, thanks for checking.
>
> On 30/08/17 16:18, Sam Skipsey wrote:
>
> Right, so the other errors are coming from the fact that the script tries to
> get a credential (conventionally in /var/lib/ldap/hostcert.pem , hostkey.pem
> , owned by ldap), which I guess doesn't exist for your install? (This can
> just be another copy of the hostcert for the SE).
>
> The DMLite puppet module does exactly that
> export X509_USER_CERT=/var/lib/ldap/hostcert.pem
> export X509_USER_KEY=/var/lib/ldap/hostkey.pem
> and there is no difference between the above and what is in
> /etc/grid-security.
> Also, this error seemed to have vanished from the logs. I can still
> reproduce it if I run
> . /var/lib/bdii/gip/provider/se-dpm
> manually.
>
> Cheers,
> Luke
>
>
> Finally, the errors from the srm publishing module (the one which isn't
> se-dpm) can probably be ignored completely, as you obviously don't have such
> a thing!
>
> Sam
>
> On Wed, Aug 30, 2017 at 2:24 PM Luke Kreczko <[log in to unmask]>
> wrote:
>>
>> Hi Sam,
>>
>>
>> AFAIK The dmlite puppet module configures /etc/bdii/bdii.conf, which looks
>> relatively simple [1].
>>
>> Looking through the log file it looks like the dpm-listspaces is crashing
>> [2], however if I execute the command mentioned in the log
>>
>> dpm-listspaces --gip --protocols --basedir home --site
>> UKI-SOUTHGRID-BRIS-HEP --glue2 --use-dmlite-conf /etc/dmlite.conf
>>
>> by hand, I do get a nicely looking response which includes
>>
>> GLUE2StorageServiceCapacityType: online
>> GLUE2StorageServiceCapacityTotalSize: 636879
>> GLUE2StorageServiceCapacityFreeSize: 64853
>> GLUE2StorageServiceCapacityUsedSize: 572026
>> GLUE2StorageServiceCapacityReservedSize: 0
>>
>> compared to
>>
>> dmlite-shell -e 'qryconf'
>> POOLS hdfs_pool DEFSIZE 0 GC_START_THRESH 0 GC_STOP_THRESH 0 DEF_LIFETIME
>> 0 DEFPINTIME 0 MAX_LIFETIME 0 MAXPINTIME 0 GROUPS [0] FSS_POLICY GC_POLICY
>> MIG_POLICY RS_POLICY RET_POLICY S_TYPE
>> 17/08/30 14:02:39 WARN util.NativeCodeLoader: Unable to load native-hadoop
>> library for your platform... using builtin-java classes where applicable
>> CAPACITY 579.24TB FREE 58.97TB ( 10.2%)
>>
>>
>> This looks to me if whatever is executing the script is calling an
>> incompatible java version [3].
>>
>> Since we have both Java 6 and 7 installed and HDFS only works with Java 7
>> I tried removing Java 6 - however it tries to pull the DMLite HDFS plugin
>> with it :(.
>>
>> After checking the java installed I realised that only Java 6 has the
>> -devel package installed - /usr/sbin/alternatives is confused by it!
>>
>>
>> So, once that was fixed the error message is much shorter:
>>
>> 17/08/30 14:18:49 WARN util.NativeCodeLoader: Unable to load native-hadoop
>> library for your platform... using builtin-java classes where applicable
>> 2017-08-30 14:18:50,419: [DEBUG] Running /var/lib/bdii/gip/provider/se-dpm
>> send2nsd: NS002 - send error : client_establish_context: Could not find or
>> use a credential
>> ERROR: unable to open DPM root (/dpm)
>> 2017-08-30 14:18:50,540: [DEBUG] Running
>> /var/lib/bdii/gip/provider/service-srm2.2
>> /usr/bin/glite-info-service-dpm: line 26: /etc/sysconfig/srmv2.2: No such
>> file or directory
>>
>> However, it seems that the GLUE information is now published (please
>> confirm).
>>
>>
>> Cheers,
>>
>> Luke
>>
>>
>> [1]
>>
>> /etc/bdii/bdii.con
>>
>> BDII_LOG_FILE=/var/log/bdii/bdii-update.log
>> BDII_LOG_LEVEL=DEBUG
>> BDII_LDIF_DIR=/var/lib/bdii/gip/ldif
>> BDII_PROVIDER_DIR=/var/lib/bdii/gip/provider
>> BDII_PLUGIN_DIR=/var/lib/bdii/gip/plugin
>> BDII_PORT=2170
>> BDII_BREATHE_TIME=120
>> BDII_READ_TIMEOUT=300
>> BDII_ARCHIVE_SIZE=0
>> BDII_DELETE_DELAY=0
>> BDII_USER=ldap
>> BDII_VAR_DIR=/var/lib/bdii
>>
>>
>> [2]
>>
>> 2017-08-30 14:04:37,113: [DEBUG] Running /var/lib/bdii/gip/provider/se-dpm
>> loadFileSystems error:
>> java.lang.UnsupportedClassVersionError: org/apache/hadoop/fs/FileSystem :
>> Unsupported major.minor version 51.0
>> at java.lang.ClassLoader.defineClass1(Native Method)
>> at java.lang.ClassLoader.defineClass(ClassLoader.java:648)
>> at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
>> at java.net.URLClassLoader.defineClass(URLClassLoader.java:272)
>> at java.net.URLClassLoader.access$000(URLClassLoader.java:68)
>> at java.net.URLClassLoader$1.run(URLClassLoader.java:207)
>> at java.net.URLClassLoader$1.run(URLClassLoader.java:201)
>> at java.security.AccessController.doPrivileged(Native Method)
>> at java.net.URLClassLoader.findClass(URLClassLoader.java:200)
>> at java.lang.ClassLoader.loadClass(ClassLoader.java:325)
>> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:296)
>> at java.lang.ClassLoader.loadClass(ClassLoader.java:270)
>> hdfsBuilderConnect(forceNewInstance=1, nn=jt-37-00.dice.priv, port=0,
>> kerbTicketCachePath=(NULL), userName=dpmmgr) error:
>> java.lang.UnsupportedClassVersionError:
>> org/apache/hadoop/conf/Configuration : Unsupported major.minor version 51.0
>> at java.lang.ClassLoader.defineClass1(Native Method)
>> at java.lang.ClassLoader.defineClass(ClassLoader.java:648)
>> at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
>> at java.net.URLClassLoader.defineClass(URLClassLoader.java:272)
>> at java.net.URLClassLoader.access$000(URLClassLoader.java:68)
>> at java.net.URLClassLoader$1.run(URLClassLoader.java:207)
>> at java.net.URLClassLoader$1.run(URLClassLoader.java:201)
>> at java.security.AccessController.doPrivileged(Native Method)
>> at java.net.URLClassLoader.findClass(URLClassLoader.java:200)
>> at java.lang.ClassLoader.loadClass(ClassLoader.java:325)
>> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:296)
>> at java.lang.ClassLoader.loadClass(ClassLoader.java:270)
>> Traceback (most recent call last):
>> File "/usr/bin/dpm-listspaces", line 1801, in <module>
>> capacity, free = dmlite.getPoolInfo(pool.poolname)
>> File "/usr/bin/dpm-listspaces", line 1503, in getPoolInfo
>> self.poolHandler = self.poolDriver.createPoolHandler(poolname)
>> pydmlite.DmException: [#01.000255] Could not create a HdfsPoolDriver:
>> cannot connect to Hdfs
>> /var/lib/bdii/gip/provider/se-dpm: line 5: 4288 Aborted
>> dpm-listspaces --gip --protocols --basedir home --site
>> UKI-SOUTHGRID-BRIS-HEP --glue2 --use-dmlite-conf /etc/dmlite.conf
>> 2017-08-30 14:04:38,036: [DEBUG] Running
>> /var/lib/bdii/gip/provider/service-srm2.2
>> /usr/bin/glite-info-service-dpm: line 26: /etc/sysconfig/srmv2.2: No such
>> file or directory
>> srmv2.2: unrecognized service
>>
>>
>> [3]
>> java -version
>> java version "1.7.0_151"
>> OpenJDK Runtime Environment (rhel-2.6.11.0.el6_9-x86_64 u151-b00)
>> OpenJDK 64-Bit Server VM (build 24.151-b00, mixed mode)
>>
>> /usr/lib/jvm/jre-1.6.0-openjdk.x86_64/bin/java -version
>> java version "1.6.0_41"
>> OpenJDK Runtime Environment (IcedTea6 1.13.13)
>> (rhel-1.13.13.1.el6_8-x86_64)
>> OpenJDK 64-Bit Server VM (build 23.41-b41, mixed mode)
>>
>> lrwxrwxrwx 1 root root 46 Aug 16 05:49 java ->
>> /usr/lib/jvm/jre-1.7.0-openjdk.x86_64/bin/java
>> lrwxrwxrwx 1 root root 48 Aug 16 05:49 java.1.gz ->
>> /usr/share/man/man1/java-java-1.7.0-openjdk.1.gz
>> lrwxrwxrwx 1 root root 48 Jan 14 2017 javac ->
>> /usr/lib/jvm/java-1.6.0-openjdk.x86_64/bin/javac
>> lrwxrwxrwx 1 root root 49 Jan 14 2017 javac.1.gz ->
>> /usr/share/man/man1/javac-java-1.6.0-openjdk.1.gz
>> lrwxrwxrwx 1 root root 50 Jan 14 2017 javadoc ->
>> /usr/lib/jvm/java-1.6.0-openjdk.x86_64/bin/javadoc
>> lrwxrwxrwx 1 root root 51 Jan 14 2017 javadoc.1.gz ->
>> /usr/share/man/man1/javadoc-java-1.6.0-openjdk.1.gz
>> lrwxrwxrwx 1 root root 48 Jan 14 2017 javah ->
>> /usr/lib/jvm/java-1.6.0-openjdk.x86_64/bin/javah
>> lrwxrwxrwx 1 root root 49 Jan 14 2017 javah.1.gz ->
>> /usr/share/man/man1/javah-java-1.6.0-openjdk.1.gz
>> lrwxrwxrwx 1 root root 48 Jan 14 2017 javap ->
>> /usr/lib/jvm/java-1.6.0-openjdk.x86_64/bin/javap
>> lrwxrwxrwx 1 root root 49 Jan 14 2017 javap.1.gz ->
>> /usr/share/man/man1/javap-java-1.6.0-openjdk.1.gz
>> lrwxrwxrwx 1 root root 38 Jan 14 2017 java_sdk ->
>> /usr/lib/jvm/java-1.6.0-openjdk.x86_64
>> lrwxrwxrwx 1 root root 38 Jan 14 2017 java_sdk_1.6.0 ->
>> /usr/lib/jvm/java-1.6.0-openjdk.x86_64
>> lrwxrwxrwx 1 root root 46 Jan 14 2017 java_sdk_1.6.0_exports ->
>> /usr/lib/jvm-exports/java-1.6.0-openjdk.x86_64
>> lrwxrwxrwx 1 root root 46 Jan 14 2017 java_sdk_exports ->
>> /usr/lib/jvm-exports/java-1.6.0-openjdk.x86_64
>> lrwxrwxrwx 1 root root 38 Jan 14 2017 java_sdk_openjdk ->
>> /usr/lib/jvm/java-1.6.0-openjdk.x86_64
>> lrwxrwxrwx 1 root root 46 Jan 14 2017 java_sdk_openjdk_exports ->
>> /usr/lib/jvm-exports/java-1.6.0-openjdk.x86_64
>>
>>
>>
>>
>>
>>
>> **************************************************
>> Dr Lukasz Kreczko
>> Research Associate
>> Department of Physics
>> Particle Physics Group
>>
>> University of Bristol
>> HH Wills Physics Lab
>> University of Bristol
>> Tyndall Avenue
>> Bristol
>> BS8 1TL
>>
>> +44 (0)117 928 8724
>> [log in to unmask]
>>
>> A top 3 UK university with leading employers (2017)
>> A top 5 UK university for research (2014 REF)
>> A world top 50 university (QS Ranking 2016)
>> **************************************************
>> ________________________________
>> From: GRIDPP2: Deployment and support of SRM and local storage management
>> <[log in to unmask]> on behalf of Sam Skipsey
>> <[log in to unmask]>
>> Sent: 30 August 2017 13:21:53
>> To: [log in to unmask]
>> Subject: Re: Help: Our SE is publishing a GlueService object, but main
>> GlueSE missing
>>
>> (Specifically, it would be useful to try executing the two provider
>> scripts in /var/lib/bdii/gip/providers/ and see what they output.)
>>
>>
>> On Wed, Aug 30, 2017 at 12:52 PM Sam Skipsey <[log in to unmask]>
>> wrote:
>>>
>>> Hi Winnie:
>>>
>>> So, the BDII stuff is configured by
>>>
>>> /etc/bdii/bdii.conf
>>>
>>> which points (usually) at
>>>
>>> /var/lib/bdii/gip
>>>
>>> as the base path for stuff.
>>>
>>> On my SE, I see in that path:
>>>
>>> glite-info-service-srm2.2.conf
>>>
>>> and the usual plugins, providers, ldif directories
>>>
>>> plugins is empty
>>>
>>> ldif contains a basic default.ldif
>>>
>>> providers contains two scripts:
>>>
>>> se-dpm service-srm2.2
>>>
>>>
>>> Obviously, as you don't have an SRM, I would guess you only should have
>>> se-dpm?
>>>
>>> Our se-dpm does what I expected it to (we discussed this in the Storage
>>> Group meeting this morning), and runs dpm-listspaces in --glue2 mode:
>>>
>>>
>>> export X509_USER_CERT=/var/lib/ldap/hostcert.pem
>>>
>>> export X509_USER_KEY=/var/lib/ldap/hostkey.pem
>>>
>>> dpm-listspaces --gip --protocols --basedir home --site
>>> UKI-SCOTGRID-GLASGOW --glue2
>>>
>>>
>>> Can you check which of these things exist on your SE?
>>>
>>>
>>> Sam
>>>
>>> On Wed, Aug 23, 2017 at 10:43 AM Winnie Lacesso
>>> <[log in to unmask]> wrote:
>>>>
>>>> Good day all,
>>>>
>>>> Not a meeting today but can request SE help anyway.
>>>>
>>>> Bristol SE doesna speak SRM. It is publishing a GlueService object, but
>>>> main
>>>> GlueSE missing. (CAVEAT: Sadly, my bdii/GLUE knowhow = .00001)
>>>>
>>>> I asked Stephen Burke (GLUE Guru) about this, he said:
>>>>
>>>> > In GLUE 1 your SE is publishing a GlueService object, but the main
>>>> > GlueSE
>>>> > is missing:
>>>> >
>>>> >ldapsearch -x -h lcgbdii02.phy.bris.ac.uk -p 2170 -b o=grid
>>>> > objectclass=GlueSE
>>>> ># extended LDIF
>>>> >#
>>>> ># LDAPv3
>>>> ># base <o=grid> with scope subtree
>>>> ># filter: objectclass=GlueSE
>>>> ># requesting: ALL
>>>> >#
>>>> >
>>>> ># search result
>>>> >search: 2
>>>> >result: 0 Success
>>>> >
>>>> ># numResponses: 1
>>>> >
>>>> >Similarly in GLUE 2:
>>>> >
>>>> >ldapsearch -x -h lcgbdii02.phy.bris.ac.uk -p 2170 -b o=glue
>>>> > objectclass=GLUE2StorageService
>>>> ># extended LDIF
>>>> >#
>>>> ># LDAPv3
>>>> ># base <o=glue> with scope subtree
>>>> ># filter: objectclass=GLUE2StorageService
>>>> ># requesting: ALL
>>>> >#
>>>> >
>>>> ># search result
>>>> >search: 2
>>>> >result: 0 Success
>>>> >
>>>> ># numResponses: 1
>>>> >
>>>> > The publishing isn't intrinsically connected to whether it's an SRM or
>>>> > not, but if no-one has written an info provider for it then obviously
>>>> > it won't be visible.
>>>>
>>>> Possibly relevant installed:
>>>>
>>>> apr-util-ldap-1.3.9-3.el6_0.1.x86_64.rpm
>>>> bdii-5.2.23-1.el6.noarch.rpm
>>>> openldap-2.4.40-6.el6_7.x86_64.rpm
>>>> openldap-clients-2.4.40-6.el6_7.x86_64.rpm
>>>> openldap-servers-2.4.40-6.el6_7.x86_64.rpm
>>>> perl-LDAP-0.40-1.el6.noarch.rpm
>>>> python-ldap-2.3.10-1.el6.x86_64.rpm
>>>>
>>>> also possibly relevant installed:
>>>>
>>>> dmlite-plugins-hdfs-0.7.8-1.20161117.1205rc.el6.x86_64.rpm
>>>> dmlite-plugins-memcache-0.8.5-1.el6.x86_64.rpm
>>>> dmlite-plugins-mysql-0.8.5-1.el6.x86_64.rpm
>>>> gstreamer-plugins-base-0.10.29-2.el6.x86_64.rpm
>>>>
>>>> hadoop-0.20-mapreduce-2.6.0+cdh5.7.1+1335-1.cdh5.7.1.p0.15.el6.x86_64.rpm
>>>> hadoop-2.6.0+cdh5.7.1+1335-1.cdh5.7.1.p0.15.el6.x86_64.rpm
>>>> hadoop-client-2.6.0+cdh5.7.1+1335-1.cdh5.7.1.p0.15.el6.x86_64.rpm
>>>> hadoop-hdfs-2.6.0+cdh5.7.1+1335-1.cdh5.7.1.p0.15.el6.x86_64.rpm
>>>> hadoop-libhdfs-2.6.0+cdh5.7.1+1335-1.cdh5.7.1.p0.15.el6.x86_64.rpm
>>>> hadoop-mapreduce-2.6.0+cdh5.7.1+1335-1.cdh5.7.1.p0.15.el6.x86_64.rpm
>>>> hadoop-yarn-2.6.0+cdh5.7.1+1335-1.cdh5.7.1.p0.15.el6.x86_64.rpm
>>>> perl-Module-Pluggable-3.90-141.el6.x86_64.rpm
>>>> xrootd-server-atlas-n2n-plugin-2.2-0.el6.x86_64.rpm
>>>>
>>>> in /etc/bdii:
>>>> -rw-r----- 1 ldap ldap 3720 Jun 8 2016 bdii-slapd.conf
>>>> -rw-r--r-- 1 root root 329 Mar 18 2015 bdii.conf
>>>> drwxr-xr-x 2 root root 4096 Mar 18 2015 gip/
>>>> -rw-r----- 1 ldap ldap 4012 Mar 18 2015 bdii-top-slapd.conf
>>>> -rw-r--r-- 1 root root 3970 Oct 13 2014 BDII.schema
>>>> -rw-r--r-- 1 root root 276 Oct 13 2014 DB_CONFIG
>>>> -rw-r--r-- 1 root root 510 Oct 13 2014 DB_CONFIG_top
>>>>
>>>> gip is empty.
>>>>
>>>> In /var/lib/bdii:
>>>> -rw-r--r-- 1 ldap ldap 0 Dec 15 2016 add.err
>>>> -rw-r--r-- 1 ldap ldap 52 Aug 23 10:36 add.ldif
>>>> drwxr-xr-x 2 ldap ldap 4096 Aug 23 10:36 archive/
>>>> drwxr-xr-x 5 ldap ldap 100 Dec 15 2016 db/
>>>> -rw-r--r-- 1 ldap ldap 0 Sep 30 2016 delete.err
>>>> -rw-r--r-- 1 ldap ldap 0 Aug 23 10:36 delete.ldif
>>>> drwxr-xr-x 5 ldap ldap 4096 Mar 18 2015 gip/
>>>> -rw-r--r-- 1 ldap ldap 0 Aug 23 10:36 modify.err
>>>> -rw-r--r-- 1 ldap ldap 82391 Aug 23 10:36 modify.ldif
>>>> -rw-r--r-- 1 ldap ldap 58419 Aug 23 10:36 new.ldif
>>>> -rw-r--r-- 1 ldap ldap 0 Aug 23 10:36 old.err
>>>> -rw-r--r-- 1 ldap ldap 2785 Aug 23 10:36 old.ldif
>>>> root@lcgse01> ls -lR gip
>>>>
>>>> ls -lR gip:
>>>> total 16
>>>> -rwxr-xr-x 1 ldap ldap 648 Mar 18 2015 glite-info-service-srm2.2.conf*
>>>> drwxr-xr-x 2 ldap ldap 4096 Mar 18 2015 ldif/
>>>> drwxr-xr-x 2 ldap ldap 4096 Oct 13 2014 plugin/
>>>> drwxr-xr-x 2 ldap ldap 4096 Oct 27 2015 provider/
>>>>
>>>> gip/ldif:
>>>> total 4
>>>> -rw-r--r-- 1 ldap ldap 439 Oct 13 2014 default.ldif
>>>>
>>>> gip/plugin:
>>>> total 0
>>>>
>>>> gip/provider:
>>>> total 8
>>>> -rwxr-xr-x 1 root root 287 Oct 27 2015 se-dpm*
>>>> -rwxr-xr-x 1 ldap ldap 146 Mar 18 2015 service-srm2.2*
>>>>
>>>> Can anyone advise how to have this dmlite-hdfs SE publish
>>>> objectclass=GlueSE in GLUE 1 & objectclass=GLUE2StorageService in GLUE
>>>> 2?
>
>
--
Sent from the pit of despair
-----------------------------------------------------------
[log in to unmask]
HEP Group/Physics Dep
Imperial College
London, SW7 2BW
Tel: +44-(0)20-75947810
http://www.hep.ph.ic.ac.uk/~dbauer/
|
