JISCMail - GRIDPP-STORAGE Archives

Hi all,

At IC we have the following dCache nodes:

admin_node --- gw04.hep.ph.ic.ac.uk

pool_node --- gw03.hep.ph.ic.ac.uk

We want to configure the pool node such that clients transfer

files directly to/from the pool node, and not via the admin node.

We installed pnfs and dcache-opt rpms on the pool node

and started pnfs and dcache-opt (including gsiftp) services.

> /opt/pnfs.3.1.10/pnfs/install/pnfs-install.sh

NB. There was no pnfs configuration (this seems to be part of the

dcache-core package, which of course we did not want to install on the

pool node!) so:

>/sbin/service pnfs start

>pnfs unrecognised service

so we used

> /opt/pnfs.3.1.10/pnfs/bin/pnfs start

then we manually created the

/pnfs/fs/usr/data/dteam

directory and set the ownership and permissions and

> ln -s /pnfs/fs/usr /pnfs/hep.ph.ic.ac.uk

We also started dcahe-opt service on the pool node

>[root@gw03 root]# /opt/d-cache/bin/dcache-opt start

>Starting dcache services: Starting gridftpdoorDomain 6 5 4 3 2 1 0 Done (pid=30168)

>Starting gsidcapdoorDomain 6 5 4 3 2 1 0 Done (pid=30250)

>Starting srmDomain 6 5 4 3 2 1 0 Done (pid=30333)

Then we conducted tests with srmcp and following are our observations.

1. If pnfs service is not running on the pool_node then it is not

selected by the dCache as a TURL. Admin_node is selected

as a TURL by default and srmcp is succesfull.

2. If pnfs service is running on the pool_node then it fails, if the TURL is selected through it.

TURL=gsiftp://gw03.hep.ph.ic.ac.uk:2811///dteam/testfile1721

// Error message

-------------------------------------------------------------------------------------------------

[root@gw04 root]# /opt/d-cache/srm/bin/srmcp -debug=true -x509_user_proxy=./mtmproxy file:////root/testfile srm://gw04.hep.ph.ic.ac.uk:8443/pnfs/hep.ph.ic.ac.uk

/data/dteam/testfile1721

SRM Configuration:

debug=true

gsissl=true

help=false

pushmode=false

userproxy=true

buffer_size=2048

tcp_buffer_size=0

config_file=/root/.srmconfig/config.xml

glue_mapfile=/opt/d-cache/srm/conf/SRMServerV1.map

webservice_path=srm/managerv1.wsdl

webservice_protocol=https

gsiftpclinet=globus-url-copy

protocols_list=http,gsiftp

save_config_file=null

srmcphome=/opt/d-cache/srm

urlcopy=/opt/d-cache/srm/bin/url-copy.sh

x509_user_cert=/root/k5-ca-proxy.pem

x509_user_key=/root/k5-ca-proxy.pem

x509_user_proxy=./mtmproxy

x509_user_trusted_certificates=/root/.globus/certificates

retry_num=20

retry_timeout=10000

wsdl_url=null

use_urlcopy_script=false

connect_to_wsdl=false

from[0]=file:////root/testfile

to=srm://gw04.hep.ph.ic.ac.uk:8443/pnfs/hep.ph.ic.ac.uk/data/dteam/testfile1721

Tue May 17 13:49:10 BST 2005: starting SRMPutClient

Tue May 17 13:49:10 BST 2005: SRMClient(https,srm/managerv1.wsdl,true)

Tue May 17 13:49:10 BST 2005: connecting to server

Tue May 17 13:49:10 BST 2005: connected to server, obtaining proxy

SRMClientV1 : connecting to srm at httpg://gw04.hep.ph.ic.ac.uk:8443/srm/managerv1

Tue May 17 13:49:13 BST 2005: got proxy of type class org.dcache.srm.client.SRMClientV1

SRMClientV1 : put, sources[0]="/root/testfile"

SRMClientV1 : put, dests[0]="srm://gw04.hep.ph.ic.ac.uk:8443/pnfs/hep.ph.ic.ac.uk/data/dteam/testfile1721"

SRMClientV1 : put, protocols[0]="http"

SRMClientV1 : put, protocols[1]="dcap"

SRMClientV1 : put, protocols[2]="gsiftp"

SRMClientV1 : put, contacting service httpg://gw04.hep.ph.ic.ac.uk:8443/srm/managerv1

doneAddingJobs is false

copy_jobs is empty

Tue May 17 13:49:17 BST 2005: srm returned requestId = -2147483536

Tue May 17 13:49:17 BST 2005: sleeping 1 seconds ...

Tue May 17 13:49:19 BST 2005: FileRequestStatus with SURL=srm://gw04.hep.ph.ic.ac.uk:8443/pnfs/hep.ph.ic.ac.uk/data/dteam/testfile1721 is Ready

Tue May 17 13:49:19 BST 2005: received TURL=gsiftp://gw03.hep.ph.ic.ac.uk:2811///dteam/testfile1721

doneAddingJobs is false

copy_jobs is not empty

copying CopyJob, source = file:////root/testfile destination = gsiftp://gw03.hep.ph.ic.ac.uk:2811///dteam/testfile1721

GridftpClient: connecting to gw03.hep.ph.ic.ac.uk on port 2811

GridftpClient: gridFTPClient tcp buffer size is set to 1048576

GridftpClient: gridFTPWrite started, source file is java.io.RandomAccessFile@140984b destination path is //dteam/testfile1721

GridftpClient: parallelism: 10

GridftpClient: adler 32 for file java.io.RandomAccessFile@140984b is 82af088a

GridftpClient: waiting for completion of transfer

GridftpClient: gridFtpWrite: starting the transfer in emode to //dteam/testfile1721

GridftpClient: DiskDataSink.close() called

-----------------------------------------------------------------------------------

The client then hangs for a large time before failing with Java errors

saying no such file or directory for /pnfs/hep.ph.ic.ac.uk/data/dteam

(which is why we manually created it on gw03).

//java error

-----------------------------------------------------------------------------------

GridftpClient: DiskDataSink.close() called

org.globus.ftp.exception.ServerException: Server refused performing the request. Custom message: (error code 1) [Nested exception message: Custom message: Unexpected reply: 553 /dteam/testfile1721: No such file or directory. : /dteam]. Nested exception is org.globus.ftp.exception.UnexpectedReplyCodeException: Custom message: Unexpected reply: 553 /dteam/testfile1721: No such file or directory. : /dteam

at org.globus.ftp.vanilla.TransferMonitor.run(TransferMonitor.java:167)

GridftpClient: transfer exception

at org.globus.ftp.vanilla.TransferMonitor.run(TransferMonitor.java:167)

GridftpClient: closing client : org.dcache.srm.util.GridftpClient$FnalGridFTPClient@1a1399

GridftpClient: closed client

copy failed with the error

at org.globus.ftp.vanilla.TransferMonitor.run(TransferMonitor.java:167)

try again

sleeping for 30000 before retrying

copying CopyJob, source = file:////root/testfile destination = gsiftp://gw03.hep.ph.ic.ac.uk:2811///dteam/testfile1721

GridftpClient: connecting to gw03.hep.ph.ic.ac.uk on port 2811

GridftpClient: gridFTPClient tcp buffer size is set to 1048576

GridftpClient: gridFTPWrite started, source file is java.io.RandomAccessFile@18e8541 destination path is //dteam/testfile1721

GridftpClient: parallelism: 10

GridftpClient: adler 32 for file java.io.RandomAccessFile@18e8541 is 82af088a

GridftpClient: waiting for completion of transfer

GridftpClient: gridFtpWrite: starting the transfer in emode to //dteam/testfile1721

GridftpClient: DiskDataSink.close() called

-----------------------------------------------------------------------------------

We're not completely clear when/how gw03 selected over gw04 for the

TURL: if pnfs is not running on one, the other is always selected, but

if pnfs is running on both the choice seems almost to depend on which

service was last restarted!

Any suggestions?

Thanks,

Mona & Owen

===========================================================

Imperial College London

High Energy Physics Department

The Blackett Laboratory

Prince Consort Road, London, SW7 2BW