|
Hi John,
Yes, it seems to be a mapping problem. I'll already wrote an own
script to create the dumps, but for the transfer itself the provided
one seemed to be simple enough, but obviously is not.
I'll try to copy it with a different method than xrdcp.
Thanks,
Marcus
On Thu, Dec 10, 2015 at 11:33 AM, John Bland <[log in to unmask]> wrote:
> Hi,
>
> I didn't bother with the supplied script and wrote my own which uses rfcp to
> add the file directly as root then dpns-chgrp set the group permissions to
> ATLAS.
>
> Setting up local mapping to use xrdcp seemed more of a faff and more likely
> to break to me.
>
> John
>
>
> On 10/12/2015 10:34, Marcus Ebert wrote:
>>
>> Hi Shaun,
>>
>> The script provided by ATLAS is copying to a local area using xrdcp. In
>> this case here, the command looks like:
>> xrdcp -d3 $HOME/bin/atlasscratchdisk-dump_20151207
>>
>> root://srm.glite.ecdf.ed.ac.uk//dpm/glite.ecdf.ed.ac.uk/home/atlas/atlasscratchdisk/dumps/dump_20151207
>>
>>
>> which is executed on srm.glite.ecdf.ed.ac.uk.
>> Just before the xrdcp command, the 2 following commands are in the
>> script which work:
>> dpns-mkdir -p /dpm/glite.ecdf.ed.ac.uk/home/atlas/atlasscratchdisk/dumps
>> dpns-setacl -m g:atlas:rwx,m:rwx
>> /dpm/glite.ecdf.ed.ac.uk/home/atlas/atlasscratchdisk/dumps
>>
>> Switching on debug information, I get:
>> [2015-12-10 10:06:37.667988 +0000][Debug ][Utility ] Creating
>> a classic copy job, from file://atlasscratchdisk-dump_20151207 to
>>
>> root://srm.glite.ecdf.ed.ac.uk:1094//dpm/glite.ecdf.ed.ac.uk/home/atlas/atlasscratchdisk/dumps/dump_20151207
>>
>> [2015-12-10 10:06:37.668048 +0000][Debug ][Utility ] Opening
>> atlasscratchdisk-dump_20151207 for reading
>> [2015-12-10 10:06:37.668092 +0000][Debug ][Utility ] Opening
>>
>> root://srm.glite.ecdf.ed.ac.uk:1094//dpm/glite.ecdf.ed.ac.uk/home/atlas/atlasscratchdisk/dumps/dump_20151207?oss.asize=101029841
>> for writing
>> [2015-12-10 10:06:37.668133 +0000][Dump ][Utility ] URL:
>>
>> root://srm.glite.ecdf.ed.ac.uk:1094//dpm/glite.ecdf.ed.ac.uk/home/atlas/atlasscratchdisk/dumps/dump_20151207?oss.asize=101029841
>>
>> [2015-12-10 10:06:37.668133 +0000][Dump ][Utility ]
>> Protocol: root
>> [2015-12-10 10:06:37.668133 +0000][Dump ][Utility ] User Name:
>> [2015-12-10 10:06:37.668133 +0000][Dump ][Utility ] Password:
>> [2015-12-10 10:06:37.668133 +0000][Dump ][Utility ] Host
>> Name:srm.glite.ecdf.ed.ac.uk
>> [2015-12-10 10:06:37.668133 +0000][Dump ][Utility ]
>> Port: 1094
>> [2015-12-10 10:06:37.668133 +0000][Dump ][Utility ]
>> Path:
>> /dpm/glite.ecdf.ed.ac.uk/home/atlas/atlasscratchdisk/dumps/dump_20151207
>> [2015-12-10 10:06:37.668187 +0000][Dump ][Utility ] URL:
>>
>> root://srm.glite.ecdf.ed.ac.uk:1094//dpm/glite.ecdf.ed.ac.uk/home/atlas/atlasscratchdisk/dumps/dump_20151207?oss.asize=101029841
>>
>> [2015-12-10 10:06:37.668187 +0000][Dump ][Utility ]
>> Protocol: root
>> [2015-12-10 10:06:37.668187 +0000][Dump ][Utility ] User Name:
>> [2015-12-10 10:06:37.668187 +0000][Dump ][Utility ] Password:
>> [2015-12-10 10:06:37.668187 +0000][Dump ][Utility ] Host
>> Name:srm.glite.ecdf.ed.ac.uk
>> [2015-12-10 10:06:37.668187 +0000][Dump ][Utility ]
>> Port: 1094
>> [2015-12-10 10:06:37.668187 +0000][Dump ][Utility ]
>> Path:
>> /dpm/glite.ecdf.ed.ac.uk/home/atlas/atlasscratchdisk/dumps/dump_20151207
>> [2015-12-10 10:06:37.668218 +0000][Debug ][File ]
>>
>> [0x1346b170@root://srm.glite.ecdf.ed.ac.uk:1094//dpm/glite.ecdf.ed.ac.uk/home/atlas/atlasscratchdisk/dumps/dump_20151207?oss.asize=101029841]
>> Sending an open command
>> [2015-12-10 10:06:37.668247 +0000][Dump ][XRootD ]
>> [srm.glite.ecdf.ed.ac.uk:1094] Sending message kXR_open (file:
>>
>> /dpm/glite.ecdf.ed.ac.uk/home/atlas/atlasscratchdisk/dumps/dump_20151207?oss.asize=101029841,
>> mode: 0644, flags: kXR_new kXR_open_updt kXR_async kXR_retstat )
>> [2015-12-10 10:06:37.668275 +0000][Dump ][PostMaster ]
>> [srm.glite.ecdf.ed.ac.uk:1094 #0] Sending message kXR_open (file:
>>
>> /dpm/glite.ecdf.ed.ac.uk/home/atlas/atlasscratchdisk/dumps/dump_20151207?oss.asize=101029841,
>> mode: 0644, flags: kXR_new kXR_open_updt kXR_async kXR_retstat )
>> (0x135e40a0) through substream 0 expecting answer at 0
>> [2015-12-10 10:06:37.668336 +0000][Dump ][AsyncSock ]
>> [srm.glite.ecdf.ed.ac.uk:1094 #0.0] Wrote a message: kXR_open (file:
>>
>> /dpm/glite.ecdf.ed.ac.uk/home/atlas/atlasscratchdisk/dumps/dump_20151207?oss.asize=101029841,
>> mode: 0644, flags: kXR_new kXR_open_updt kXR_async kXR_retstat )
>> (0x135e40a0), 116 bytes
>> [2015-12-10 10:06:37.668364 +0000][Dump ][AsyncSock ]
>> [srm.glite.ecdf.ed.ac.uk:1094 #0.0] Successfully sent message: kXR_open
>> (file:
>>
>> /dpm/glite.ecdf.ed.ac.uk/home/atlas/atlasscratchdisk/dumps/dump_20151207?oss.asize=101029841,
>> mode: 0644, flags: kXR_new kXR_open_updt kXR_async kXR_retstat )
>> (0x135e40a0).
>> [2015-12-10 10:06:37.668381 +0000][Dump ][XRootD ]
>> [srm.glite.ecdf.ed.ac.uk:1094] Message kXR_open (file:
>>
>> /dpm/glite.ecdf.ed.ac.uk/home/atlas/atlasscratchdisk/dumps/dump_20151207?oss.asize=101029841,
>> mode: 0644, flags: kXR_new kXR_open_updt kXR_async kXR_retstat ) has
>> been successfully sent.
>> [2015-12-10 10:06:37.668407 +0000][Dump ][PostMaster ]
>> [srm.glite.ecdf.ed.ac.uk:1094 #0.0] Nothing to write, disable uplink
>> [2015-12-10 10:06:37.682412 +0000][Dump ][XRootDTransport ] [msg:
>> 0x135e3f40] Expecting 22 bytes of message body
>> [2015-12-10 10:06:37.682451 +0000][Dump ][AsyncSock ]
>> [srm.glite.ecdf.ed.ac.uk:1094 #0.0] Received message header for
>> 0x135e3f40 size: 8
>> [2015-12-10 10:06:37.682477 +0000][Dump ][AsyncSock ]
>> [srm.glite.ecdf.ed.ac.uk:1094 #0.0] Received message 0x135e3f40 of 30
>> bytes
>> [2015-12-10 10:06:37.682494 +0000][Dump ][PostMaster ]
>> [srm.glite.ecdf.ed.ac.uk:1094 #0] Handling received message: 0x135e3f40.
>> [2015-12-10 10:06:37.682532 +0000][Dump ][XRootD ]
>> [srm.glite.ecdf.ed.ac.uk:1094] Got a kXR_error response to request
>> kXR_open (file:
>>
>> /dpm/glite.ecdf.ed.ac.uk/home/atlas/atlasscratchdisk/dumps/dump_20151207?oss.asize=101029841,
>> mode: 0644, flags: kXR_new kXR_open_updt kXR_async kXR_retstat ) [3010]
>> Permission denied
>> [2015-12-10 10:06:37.682566 +0000][Debug ][XRootD ]
>> [srm.glite.ecdf.ed.ac.uk:1094] Handling error while processing kXR_open
>> (file:
>>
>> /dpm/glite.ecdf.ed.ac.uk/home/atlas/atlasscratchdisk/dumps/dump_20151207?oss.asize=101029841,
>> mode: 0644, flags: kXR_new kXR_open_updt kXR_async kXR_retstat ):
>> [ERROR] Error response: Unknown error 3010.
>> [2015-12-10 10:06:37.682615 +0000][Debug ][File ]
>>
>> [0x1346b170@root://srm.glite.ecdf.ed.ac.uk:1094//dpm/glite.ecdf.ed.ac.uk/home/atlas/atlasscratchdisk/dumps/dump_20151207?oss.asize=101029841]
>> Open has returned with status [ERROR] Server responded with an error:
>> [3010] Permission denied
>> [2015-12-10 10:06:37.682640 +0000][Debug ][File ]
>>
>> [0x1346b170@root://srm.glite.ecdf.ed.ac.uk:1094//dpm/glite.ecdf.ed.ac.uk/home/atlas/atlasscratchdisk/dumps/dump_20151207?oss.asize=101029841]
>> Error while opening at srm.glite.ecdf.ed.ac.uk:1094: [ERROR] Server
>> responded with an error: [3010] Permission denied
>>
>> It looks like that the directory created by dpns-mkdir is different than
>> what xrootd is trying to write to. The copy tries to put the file in
>>
>> root://srm.glite.ecdf.ed.ac.uk:1094//dpm/glite.ecdf.ed.ac.uk/home/atlas/atlasscratchdisk/dumps
>>
>> but that doesn't exists while the one created with dpns-mkdir is still
>> there:
>> [root@srm bin]# xrd srm.glite.ecdf.ed.ac.uk ls
>> /dpm/glite.ecdf.ed.ac.uk/home/atlas/atlasscratchdisk/dumps
>>
>> [root@srm bin]#
>>
>> (just empty, doesn't exist)
>> [root@srm bin]# xrd srm.glite.ecdf.ed.ac.uk ls /
>> dr-x(019) 0 2015-07-21 13:36:23 /atlas
>> dr-x(019) 0 2012-01-05 22:12:09 /cms
>> dr-x(019) 0 2014-11-20 11:46:19 /dteam
>> dr-x(019) 0 2008-08-01 03:42:24 /gridpp
>> dr-x(019) 0 2010-05-21 08:24:53 /hone
>> dr-x(019) 0 2011-03-02 20:10:51 /ilc
>> dr-x(019) 0 2011-07-11 17:55:57 /ildg
>> dr-x(019) 0 2011-01-11 11:09:14 /lhcb
>> dr-x(019) 0 2015-12-10 10:26:37 /ops
>> dr-x(019) 0 2008-07-01 03:42:24 /vo.gridpp.ac.uk
>> dr-x(019) 0 2008-11-01 04:42:32 /vo.nanocmos.ac.uk
>> dr-x(019) 0 2008-05-01 03:42:22 /vo.scotgrid.ac.uk
>>
>> [root@srm bin]# xrd srm.glite.ecdf.ed.ac.uk ls
>> /atlas/atlasscratchdisk/dumps
>>
>> [root@srm bin]# xrd srm.glite.ecdf.ed.ac.uk ls /atlas/atlasscratchdisk/
>> dr-x(019) 0 2015-12-10 10:25:24 /atlas/atlasscratchdisk/SAM
>> dr-x(019) 0 2015-11-09 17:56:40 /atlas/atlasscratchdisk/rucio
>> dr-x(019) 0 2011-09-06 21:04:18
>> /atlas/atlasscratchdisk/user.elenao
>> dr-x(019) 0 2011-05-09 03:30:46
>> /atlas/atlasscratchdisk/user.mpedraza
>> dr-x(019) 0 2010-09-12 10:38:40
>> /atlas/atlasscratchdisk/user.picazio
>> -r--(016) 5 2015-11-18 22:44:15 /atlas/atlasscratchdisk/wahtest
>>
>> Not sure if there is anything special on our site that this is different
>> and doesn't work. But would be good to see how the above looks on other
>> sites.
>>
>> Cheers,
>> Marcus
>>
>> On Thu, 10 Dec 2015, [log in to unmask] wrote:
>>
>>> Hi Marcus
>>>
>>> Not knowing a bit more it's difficult to say. I don't know where you
>>> are copying from and to. The problem may be at the destination site.
>>> Assuming you are the source site...
>>> 1. Try copying the file to some local disk. If you still get the
>>> permission denied, the problem is on you side; if not you will need to
>>> contact the destination site
>>> 2. Assuming the problem is at your side (which I doubt) try running
>>> xrdcp -d 3 .... which will produce reams and reams of output some of
>>> which may be useful
>>>
>>> Shaun
>>>
>>> -----Original Message-----
>>> From: GRIDPP2: Deployment and support of SRM and local storage
>>> management [mailto:[log in to unmask]] On Behalf Of Marcus
>>> Ebert
>>> Sent: 09 December 2015 10:10
>>> To: [log in to unmask]
>>> Subject: problem with Atlas dump
>>>
>>> Hi all,
>>>
>>> maybe this is the best place to ask for it, let me know if not:
>>> When creating the Atlas dumps, everything works except the final xrdcp
>>> which results in (was from Monday):
>>> Creating proxy ......................... Done
>>>
>>> Your proxy is valid until Tue Dec 8 21:35:04 2015
>>> [0B/0B][100%][==================================================][0B/s]
>>> Run: [ERROR] Server responded with an error: [3010] Permission denied
>>>
>>> [0B/0B][100%][==================================================][0B/s]
>>> Run: [ERROR] Server responded with an error: [3010] Permission denied
>>>
>>> [0B/0B][100%][==================================================][0B/s]
>>> Run: [ERROR] Server responded with an error: [3010] Permission denied
>>>
>>> The mapping to Atlas was done as it is mentioned at the beginning of
>>> the provided script, and also the creating of the directory and
>>> changing acl just before the xrdcp worked, only the xrdcp fails.
>>>
>>> Has anyone else seen this on another site or knows the reason for it?
>>>
>>> Cheers,
>>> Marcus
>>>
>>> --
>>> The University of Edinburgh is a charitable body, registered in
>>> Scotland, with registration number SC005336.
>>>
>>
>
>
> --
> John Bland [log in to unmask]
> Research Fellow office: 220
> High Energy Physics Division tel (int): 42911
> Oliver Lodge Laboratory tel (ext): +44 (0)151 794 2911
> University of Liverpool http://www.liv.ac.uk/physics/hep/
> "I canna change the laws of physics, Captain!"
>
--
The University of Edinburgh is a charitable body, registered in
Scotland, with registration number SC005336.
|
