Hi Brian,
I changed my default group, so that I could cd into the directory structure and
I can do that. So there should be no reason for this to happen.
When I transfered the other data, the group that I transfered was not my default
group, it was just one of my secondary groups.
I have also done a test transfer for the user that I did before, with me in the
new group, and that worked just fine.
But the group change should only have a positive effect, namely allow me
visibility in the directory, surely.
I am now back in my old group, the directory structure is fully visible to me -
so why would that not work?
Lydia
On Tue, 7 Jul 2015, [log in to unmask] wrote:
> You should be able to this page showing the FTS job:
> https://lcgfts3.gridpp.rl.ac.uk:8449/fts3/ftsmon/#/job/32164cff-0ceb-48eb-a333-3c0c35e66de3
>
> The error message is that
>
> Tue Jul 7 11:56:52 2015 INFO SOURCE Stat the source surl start
> Tue Jul 7 11:56:53 2015 WARNING SOURCE Stat failed with globus_ftp_client:
the server responded with an error 500 500-Command failed :
globus_i_gfs_data.c:globus_i_gfs_data_check_path:1644: 500-Path not allowed. 500 End. (70)
> Tue Jul 7 11:56:53 2015 WARNING SOURCE Stat the file will be retried
> Tue Jul 7 11:56:56 2015 WARNING SOURCE Stat failed with globus_ftp_client: the server responded with an error 500 500-Command failed : globus_i_gfs_data.c:globus_i_gfs_data_check_path:1644: 500-Path not allowed. 500 End. (70)
> Tue Jul 7 11:56:56 2015 WARNING SOURCE Stat the file will be retried
> Tue Jul 7 11:56:59 2015 WARNING SOURCE Stat failed with globus_ftp_client: the server responded with an error 500 500-Command failed : globus_i_gfs_data.c:globus_i_gfs_data_check_path:1644: 500-Path not allowed. 500 End. (70)
> Tue Jul 7 11:56:59 2015 WARNING SOURCE Stat the file will be retried
> Tue Jul 7 11:57:02 2015 WARNING SOURCE Stat failed with globus_ftp_client: the server responded with an error 500 500-Command failed : globus_i_gfs_data.c:globus_i_gfs_data_check_path:1644: 500-Path not allowed. 500 End. (70)
> Tue Jul 7 11:57:02 2015 WARNING SOURCE Stat the file will be retried
> Tue Jul 7 11:57:05 2015 ERROR No more retries for stat the file
> Tue Jul 7 11:57:05 2015 ERROR SOURCE Failed to get source file size, errno:70, globus_ftp_client: the server responded with an error 500 500-Command failed : globus_i_gfs_data.c:globus_i_gfs_data_check_path:1644: 500-Path not allowed. 500 End.
> Tue Jul 7 11:57:05 2015 INFO Report FAILED back to the server
> Is this due to you r change in group?
> Brian
>
> And here is an example of a log file in full :
> Tue Jul 7 11:56:52 2015 INFO Send monitoring start message
> Tue Jul 7 11:56:52 2015 INFO Start message content: ST {"$a$":"lcgfts3.gridpp.rl.ac.uk","$b$":"2015-07-07-1056__data.cosma.dur.ac.uk__srm-dirac.gridpp.rl.ac.uk__370482000__32164cff-0ceb-48eb-a333-3c0c35e66de3","$c$":"lcgfts3.gridpp.rl.ac.uk","$d$":"1436266612000.000000","$e$":"","$f$":"2.2.0","$g$":"nil","$h$":"gsiftp://data.cosma.dur.ac.uk/cosma5/data/dp010/dc-hood1/manyloopsmedres/23loopsmed.err","$i$":"srm://srm-dirac.gridpp.rl.ac.uk/castor/ads.rl.ac.uk/prod/vo.dirac.ac.uk/DiRAC/tape/durham.ac.uk/cosma5/data/dp010/dc-hood1/manyloopsmedres/23loopsmed.err","$j$":"data.cosma.dur.ac.uk","$k$":"srm-dirac.gridpp.rl.ac.uk","$l$":"","$m$":"","$n$":"gsiftp://data.cosma.dur.ac.uk__srm://srm-dirac.gridpp.rl.ac.uk","$o$":"","$p$":"","$q$":"/C=UK/O=eScience/OU=Durham/L=eScience/CN=lydia heck/CN=Robot:GridClient","$r$":"","$s$":""}
> Tue Jul 7 11:56:52 2015 INFO Transfer accepted
> Tue Jul 7 11:56:52 2015 INFO Proxy:/tmp/x509up_h7332377123696962123XcXukXoXescienceXouXdurhamXlXescienceXcnXlydiaXheckXcnXrobotXgridclient
> Tue Jul 7 11:56:52 2015 INFO User DN:/C=UK/O=eScience/OU=Durham/L=eScience/CN=lydia heck/CN=Robot:GridClient
> Tue Jul 7 11:56:52 2015 INFO VO:nil
> Tue Jul 7 11:56:52 2015 INFO Job id:32164cff-0ceb-48eb-a333-3c0c35e66de3
> Tue Jul 7 11:56:52 2015 INFO File id:370482000
> Tue Jul 7 11:56:52 2015 INFO Source url:gsiftp://data.cosma.dur.ac.uk/cosma5/data/dp010/dc-hood1/manyloopsmedres/23loopsmed.err
> Tue Jul 7 11:56:52 2015 INFO Dest url:srm://srm-dirac.gridpp.rl.ac.uk/castor/ads.rl.ac.uk/prod/vo.dirac.ac.uk/DiRAC/tape/durham.ac.uk/cosma5/data/dp010/dc-hood1/manyloopsmedres/23loopsmed.err
> Tue Jul 7 11:56:52 2015 INFO Overwrite enabled:0
> Tue Jul 7 11:56:52 2015 INFO Dest space token:
> Tue Jul 7 11:56:52 2015 INFO Source space token:
> Tue Jul 7 11:56:52 2015 INFO Pin lifetime:-1
> Tue Jul 7 11:56:52 2015 INFO BringOnline:-1
> Tue Jul 7 11:56:52 2015 INFO Checksum:
> Tue Jul 7 11:56:52 2015 INFO Checksum enabled:No checksum comparison
> Tue Jul 7 11:56:52 2015 INFO User filesize:0
> Tue Jul 7 11:56:52 2015 INFO File metadata:
> Tue Jul 7 11:56:52 2015 INFO Job metadata:
> Tue Jul 7 11:56:52 2015 INFO Bringonline token:
> Tue Jul 7 11:56:52 2015 INFO Multihop: 0
> Tue Jul 7 11:56:52 2015 INFO UDT: 0
> Tue Jul 7 11:56:52 2015 INFO Active: 0
> Tue Jul 7 11:56:52 2015 INFO Debug level: 0
> Tue Jul 7 11:56:52 2015 INFO BDII:lcgbdii.gridpp.rl.ac.uk:2170
> Tue Jul 7 11:56:52 2015 INFO Checksum timeout 0
> Tue Jul 7 11:56:52 2015 INFO SOURCE Stat the source surl start
> Tue Jul 7 11:56:53 2015 WARNING SOURCE Stat failed with globus_ftp_client: the server responded with an error 500 500-Command failed : globus_i_gfs_data.c:globus_i_gfs_data_check_path:1644: 500-Path not allowed. 500 End. (70)
> Tue Jul 7 11:56:53 2015 WARNING SOURCE Stat the file will be retried
> Tue Jul 7 11:56:56 2015 WARNING SOURCE Stat failed with globus_ftp_client: the server responded with an error 500 500-Command failed : globus_i_gfs_data.c:globus_i_gfs_data_check_path:1644: 500-Path not allowed. 500 End. (70)
> Tue Jul 7 11:56:56 2015 WARNING SOURCE Stat the file will be retried
> Tue Jul 7 11:56:59 2015 WARNING SOURCE Stat failed with globus_ftp_client: the server responded with an error 500 500-Command failed : globus_i_gfs_data.c:globus_i_gfs_data_check_path:1644: 500-Path not allowed. 500 End. (70)
> Tue Jul 7 11:56:59 2015 WARNING SOURCE Stat the file will be retried
> Tue Jul 7 11:57:02 2015 WARNING SOURCE Stat failed with globus_ftp_client: the server responded with an error 500 500-Command failed : globus_i_gfs_data.c:globus_i_gfs_data_check_path:1644: 500-Path not allowed. 500 End. (70)
> Tue Jul 7 11:57:02 2015 WARNING SOURCE Stat the file will be retried
> Tue Jul 7 11:57:05 2015 ERROR No more retries for stat the file
> Tue Jul 7 11:57:05 2015 ERROR SOURCE Failed to get source file size, errno:70, globus_ftp_client: the server responded with an error 500 500-Command failed : globus_i_gfs_data.c:globus_i_gfs_data_check_path:1644: 500-Path not allowed. 500 End.
> Tue Jul 7 11:57:05 2015 INFO Report FAILED back to the server
> Tue Jul 7 11:57:05 2015 INFO Send monitoring complete message
> Tue Jul 7 11:57:05 2015 INFO Complete message content: CO {"$a$":"2015-07-07-1056__data.cosma.dur.ac.uk__srm-dirac.gridpp.rl.ac.uk__370482000__32164cff-0ceb-48eb-a333-3c0c35e66de3","$b$":"lcgfts3.gridpp.rl.ac.uk","$c$":"","$d$":"2.2.0","$e$":"nil","$f$":"gsiftp://data.cosma.dur.ac.uk/cosma5/data/dp010/dc-hood1/manyloopsmedres/23loopsmed.err","$g$":"srm://srm-dirac.gridpp.rl.ac.uk/castor/ads.rl.ac.uk/prod/vo.dirac.ac.uk/DiRAC/tape/durham.ac.uk/cosma5/data/dp010/dc-hood1/manyloopsmedres/23loopsmed.err","$h$":"data.cosma.dur.ac.uk","$i$":"srm-dirac.gridpp.rl.ac.uk","$j$":"","$k$":"","$l$":"gsiftp://data.cosma.dur.ac.uk__srm://srm-dirac.gridpp.rl.ac.uk","$m$":"","$n$":"","$o$":"","$p$":"","$q$":"","$r$":"","$s$":"","$t$":"0","$u$":"500","$v$":"SOURCE","$w$":"TRANSFER_PREPARATION","$x$":"COMMUNICATION_ERROR_ON_SEND","$y$":"Error","$z$":"","$0$":"","$1$":"","$2$":"","$3$":"0","$4$":"","$5$":"","$6$":"","$7$":"","$8$":"","$9$":"","$10$":"","$11$":"SOURCE Failed to get source!
file size: globus_ftp_client: the server responded with an error 500 500-Command failed : globus_i_gfs_data.c:globus_i_gfs_data_check_path:1644: 500-Path not allowed. 500 End. ","$12$":"1436266612000.000000","$13$":"1436266625000.000000","$14$":"urlcopy","$15$":"/C=UK/O=eScience/OU=Durham/L=eScience/CN=lydia heck/CN=Robot:GridClient","$16$":"","$17$":"","$18$":"3","$19$":"3","$20$":"false","$21$":"UNKNOWN"}
> Tue Jul 7 11:57:06 2015 INFO Closing the log stream
>
> -----Original Message-----
> From: DiRAC Users [mailto:[log in to unmask]] On Behalf Of Lydia Heck
> Sent: 07 July 2015 11:57
> To: [log in to unmask]
> Subject: Re: Data transfers?
>
> And finally fails : I have --retry 3
>
> Lydia
>
>
>
> On Tue, 7 Jul 2015, Lydia Heck wrote:
>
>>
>> Hi Brian and Jens,
>>
>> I started yet another transfer this morning for which I had to change
>> my default group - that should not really make a difference.
>>
>> However there seems to be no transfer going ...
>>
>> On the transfer machine I see:
>>
>> # Using endpoint :
>> https://lcgfts3.gridpp.rl.ac.uk:8443/services/FileTransfer
>> # Service version : 3.7.6-1
>> # Interface version : 3.7.0
>> # Schema version : 3.5.0
>> # Service features : glite-data-fts-service-3.7.6-1 # Client version :
>> 3.2.32 # Client interface version : 3.2.32 Request ID:
>> 32164cff-0ceb-48eb-a333-3c0c35e66de3
>> Status: ACTIVE
>> Client DN: /C=UK/O=eScience/OU=Durham/L=eScience/CN=lydia
>> heck/CN=Robot:GridClient
>> Reason: <None>
>> Submission time: 2015-07-07 11:47:35
>> Files: 926
>> Priority: 3
>> VOName: nil
>> Active: 14
>> Ready: 0
>> Canceled: 0
>> Finished: 0
>> Submitted: 912
>> Failed: 0
>> Staging: 0
>> Started: 0
>> Delete: 0
>>
>>
>> then on the next one I see
>>
>> # Using endpoint :
>> https://lcgfts3.gridpp.rl.ac.uk:8443/services/FileTransfer
>> # Service version : 3.7.6-1
>> # Interface version : 3.7.0
>> # Schema version : 3.5.0
>> # Service features : glite-data-fts-service-3.7.6-1 # Client version :
>> 3.2.32 # Client interface version : 3.2.32 Request ID:
>> 32164cff-0ceb-48eb-a333-3c0c35e66de3
>> Status: ACTIVE
>> Client DN: /C=UK/O=eScience/OU=Durham/L=eScience/CN=lydia
>> heck/CN=Robot:GridClient
>> Reason: <None>
>> Submission time: 2015-07-07 11:47:35
>> Files: 926
>> Priority: 3
>> VOName: nil
>> Active: 0
>> Ready: 0
>> Canceled: 0
>> Finished: 0
>> Submitted: 926
>> Failed: 0
>> Staging: 0
>> Started: 0
>> Delete: 0
>>
>> and then
>>
>> # Using endpoint :
>> https://lcgfts3.gridpp.rl.ac.uk:8443/services/FileTransfer
>> # Service version : 3.7.6-1
>> # Interface version : 3.7.0
>> # Schema version : 3.5.0
>> # Service features : glite-data-fts-service-3.7.6-1 # Client version :
>> 3.2.32 # Client interface version : 3.2.32 Request ID:
>> 32164cff-0ceb-48eb-a333-3c0c35e66de3
>> Status: ACTIVE
>> Client DN: /C=UK/O=eScience/OU=Durham/L=eScience/CN=lydia
>> heck/CN=Robot:GridClient
>> Reason: <None>
>> Submission time: 2015-07-07 11:47:35
>> Files: 926
>> Priority: 3
>> VOName: nil
>> Active: 11
>> Ready: 0
>> Canceled: 0
>> Finished: 0
>> Submitted: 915
>> Failed: 0
>> Staging: 0
>> Started: 0
>> Delete: 0
>>
>>
>> So the system attempts the transfer, but then gives up and starts again.
>>
>> Any idea?
>>
>> Lydia
>>
>>
>>
>>
>>
>> On Fri, 3 Jul 2015, Brian Davies wrote:
>>
>>> FYI robot certificate in gridmapfile at RAL since 1300 today so
>>> trnasfers using should work.
>>>
>>>
>>> -----Original Message-----
>>> From: DiRAC Users [mailto:[log in to unmask]] On Behalf Of
>>> Jens Jensen
>>> Sent: 03 July 2015 14:21
>>> To: [log in to unmask]
>>> Subject: Re: Data transfers?
>>>
>>> Ah, first try deleting the "Bag attribute" - the first line of the
>>> certificate file should be:
>>> -----BEGIN CERTIFICATE-----
>>> ...
>>>
>>> If it still doesn't work, try stat' ing the file to see if the atime
>>> is updated, whether fts-transfer-submit is reading it.
>>>
>>> Cheers
>>> -j
>>>
>>> On 03/07/2015 14:01, Lydia Heck wrote:
>>>>
>>>> my script looks like:
>>>>
>>>>
>>>> #!/bin/bash -l
>>>> export X509_USER_CERT=$HOME/.globus/usercert-robot.pem
>>>> export X509_USER_KEY=$HOME/.globus/userkey-robot.pem
>>>>
>>>> if [[ $# == 0 ]]; then
>>>> echo "usage: submit snapdir-number"
>>>> exit 1
>>>> fi
>>>> file jch-L800-snapdir_$1
>>>> fts-transfer-submit -s \
>>>> https://lcgfts3.gridpp.rl.ac.uk:8443/services/FileTransfer -v
>>>> --retry
>>>> 3 \
>>>> -f jch-L800-snapdir_$1
>>>>
>>>>
>>>> with
>>>> $HOME/.globus/usercert-robot.pem starting with
>>>>
>>>> Bag Attributes
>>>> friendlyName: elh-cosma-cert-robot
>>>> localKeyID: 54 69 6D 65 20 31 34 33 35 37 36 30 36 36 39 33 35
>>>> 32 subject=/C=UK/O=eScience/OU=Durham/L=eScience/CN=lydia
>>>> heck/CN=Robot:GridClient
>>>> issuer=/C=UK/O=eScienceCA/OU=Authority/CN=UK e-Science CA 2B
>>>> -----BEGIN CERTIFICATE----- ......
>>>>
>>>>
>>>> but when I run the script (submit a transfer) I get:
>>>>
>>>> jch-L800-snapdir_207: ASCII text
>>>> # Using endpoint :
>>>> https://lcgfts3.gridpp.rl.ac.uk:8443/services/FileTransfer
>>>> # Service version : 3.7.6-1
>>>> # Interface version : 3.7.0
>>>> # Schema version : 3.5.0
>>>> # Service features : glite-data-fts-service-3.7.6-1 # Client version :
>>>> 3.2.32 # Client interface version : 3.2.32 Remaining time for the
>>>> local proxy is: -27hours and 0 minutes.
>>>> No proxy found on server. Requesting standard delegation.
>>>> Your local proxy has less than 1 minute to run, Please renew it
>>>> before submitting a job.
>>>>
>>>>
>>>> with edg-mkgridmap giving
>>>>
>>>> [root@data etc]# edg-mkgridmap
>>>> "/C=UK/O=eScience/OU=CLRC/L=RAL/CN=brian davies" dirac01
>>>> "/C=UK/O=eScience/OU=CLRC/L=RAL/CN=jens sha2 jensen" dirac01
>>>> "/C=UK/O=eScience/OU=Durham/L=eScience/CN=lydia heck" dirac01
>>>> "/C=UK/O=eScience/OU=Durham/L=eScience/CN=lydia
>>>> heck/CN=Robot:GridClient" dirac01
>>>>
>>>>
>>>> Any idea?
>>>>
>>>> Best wishes,
>>>> Lydia
>>>>
>>>>
>>>>
>>>>
>>>> On Fri, 3 Jul 2015, Jens Jensen wrote:
>>>>
>>>>> On 03/07/2015 11:51, Lydia Heck wrote:
>>>>>> Do I have to run a voms-proxy-init ?
>>>>> No, it should work straight off your robot certificate. The proxy
>>>>> is not needed because the private key doesn't have a passphrase:
>>>>> and we are working without the VOMS attributes for now so the
>>>>> voms-proxy itself isn't needed.
>>>>>
>>>>> So it *should* work with just the certificate alone.
>>>>>
>>>>> Cheers
>>>>> --jens
>>>>>
>>>
>>
>
|
