Hi Derek,
Thanks for the advice.
We have tried a few permutations on this as follows:
Ross, D (Derek) wrote:
> You don't need pnfs installed on the pool node. There should only be
> one pnfs running in your entire dcache instance. What you need to do
> is nfs mount the pnfs service from your admin node on the pool node.
>
Ok. We stopped pnfs on the pool node.
> On the admin node, go to /pnfs/fs/admin/etc/exports and copy the
> file 127.0.0.1 to a file that has the ip address of your pool node as
> the name, then cd into the trusted directory and do the same.
Already done! We don't like pnfs being visible to the whole world...
> Then on the pool node add something like gw04.hep.ph.ic.ac.uk:/fs
> /pnfs/fs nfs hard,intr,rw,noac,auto 0 0 to your
> /etc/fstab, create the /pnfs/fs directory, mount it and create the
> symlink
Done.
> Then, to run a gridftp door on your pool node: Set the
> /opt/d-cache/etc/door_config file on the pool node to
>
> ADMIN_NODE gw04.hep.ph.ic.ac.uk
>
> door active -------------------- GSIDCAP no GRIDFTP
> yes SRM no
>
> and then run /opt/d-cache/install/install_doors.sh and then start
> dcache-opt.
Done.
> dCache uses the name of things to work out which one to use. so if you
>have two gridftp doors named GFTP then the one that started last gets
>used. What the install_doors script does is change the name to >something
>more unique (GFTP-<hostname -s> basically) so you don't get name
>collisions. You can change the names yourself in the
>/opt/d-cache/config/*.batch files
OK - the door gridftpdoor-gw03Domain started on the pool node.
However, after this transfers still seemed to go through gw04 (admin node).
So, we stopped dcache-opt on the admin node and edited
/opt/d-cache/etc/door-config on the admin node to something like:
ADMIN_NODE gw04.hep.ph.ic.ac.uk
door active
--------------------
GSIDCAP yes
GRIDFTP no
SRM yes
ran install_doors.sh and started dcache-opt.
Now all transfers reliably go direct to gw03 (pool node).
However, they still fail!
Here is the error:
> copying CopyJob, source = file:////root/testfile destination = gsiftp://gw03.hep
> .ph.ic.ac.uk:2811//pnfs/hep.ph.ic.ac.uk/data/dteam/testfile1526
> GridftpClient: connecting to gw03.hep.ph.ic.ac.uk on port 2811
> GridftpClient: gridFTPClient tcp buffer size is set to 1048576
> GridftpClient: gridFTPWrite started, source file is java.io.RandomAccessFile@1ce
> 669e destination path is /pnfs/hep.ph.ic.ac.uk/data/dteam/testfile1526
> GridftpClient: parallelism: 10
> GridftpClient: adler 32 for file java.io.RandomAccessFile@1ce669e is 77e90756
> GridftpClient: waiting for completion of transfer
> GridftpClient: gridFtpWrite: starting the transfer in emode to /pnfs/hep.ph.ic.a
> c.uk/data/dteam/testfile1526
> org.globus.ftp.exception.ServerException: Server refused performing the request.
> Custom message: (error code 1) [Nested exception message: Custom message: Une
> xpected reply: 553 /pnfs/hep.ph.ic.ac.uk/data/dteam/testfile1526: Cannot create
> file: CacheException(rc=666;msg=Path do not exist)]. Nested exception is org.gl
> obus.ftp.exception.UnexpectedReplyCodeException: Custom message: Unexpected rep
> ly: 553 /pnfs/hep.ph.ic.ac.uk/data/dteam/testfile1526: Cannot create file: Cache
> Exception(rc=666;msg=Path do not exist)
> at org.globus.ftp.vanilla.TransferMonitor.run(TransferMonitor.java:167)
> GridftpClient: transfer exception
> org.globus.ftp.exception.ServerException: Server refused performing the request.
> Custom message: (error code 1) [Nested exception message: Custom message: Une
> xpected reply: 553 /pnfs/hep.ph.ic.ac.uk/data/dteam/testfile1526: Cannot create
> file: CacheException(rc=666;msg=Path do not exist)]. Nested exception is org.gl
> obus.ftp.exception.UnexpectedReplyCodeException: Custom message: Unexpected rep
> ly: 553 /pnfs/hep.ph.ic.ac.uk/data/dteam/testfile1526: Cannot create file: Cache
> Exception(rc=666;msg=Path do not exist)
> at org.globus.ftp.vanilla.TransferMonitor.run(TransferMonitor.java:167)
> GridftpClient: closing client : org.dcache.srm.util.GridftpClient$FnalGridFTPCli
> ent@14d5bc9
> GridftpClient: closed client
> copy failed with the error
> org.globus.ftp.exception.ServerException: Server refused performing the request.
> Custom message: (error code 1) [Nested exception message: Custom message: Une
> xpected reply: 553 /pnfs/hep.ph.ic.ac.uk/data/dteam/testfile1526: Cannot create
> file: CacheException(rc=666;msg=Path do not exist)]. Nested exception is org.gl
> obus.ftp.exception.UnexpectedReplyCodeException: Custom message: Unexpected rep
> ly: 553 /pnfs/hep.ph.ic.ac.uk/data/dteam/testfile1526: Cannot create file: Cache
> Exception(rc=666;msg=Path do not exist)
> at org.globus.ftp.vanilla.TransferMonitor.run(TransferMonitor.java:167)
> try again
which keeps repeating as the srmcp gets retried.
BTW: we also reset the gridftp root path back to "/" in case this was
causing the 'Path do not exist' error, but this had no effect.
BTW2: After running install_doors.sh on the admin node, the nice
cellInfo webpage on the admin node is showing Offline for the dcap, gftp
and srm cells, presumably as these are now dcap-gw04 and srm-gw04....
Any suggestions?
Mona and Owen.
--
=======================================================
Dr O J E Maroney # London Tier 2 Technical Co-ordinator
Tel. (+44)20 759 47802
Imperial College London
High Energy Physics Department
The Blackett Laboratory
Prince Consort Road, London, SW7 2BW
====================================
|