On Fri, Jun 16, 2006 at 09:03:15AM +0200 or thereabouts, Graeme Stewart wrote:
> Hi Steve
>
> I poked around last night and could reproduce the bug with DPM too.
Graeme,
I'll add this to the ggus ticket, I do think it is bonkers this in
GGUS but it is what we have been asked to do.
https://gus.fzk.de/pages/ticket_details.php?ticket=9358
>
> It shouldn't happen, of course. lcg-utils does do an
> srmGetFileMetadata call, as it should, but it ignores the filetype
> mask which is returned.
>
> Doing an srm-get-metadata by hand gives:
>
> ui1-gla:~$ srm-get-metadata -debug=false srm://se2-gla.scotgrid.ac.uk:
> 8443/dpm/scotgrid.ac.uk/home/dteam/lesc-transfer-test
> ...
> permMode :16893
> ...
>
> which is 100664 in octal (dunno what the top bit is for).
>
> On a directory
>
> ui1-gla:~$ srm-get-metadata -debug=false srm://se2-gla.scotgrid.ac.uk:
> 8443/dpm/scotgrid.ac.uk/home/dteam
> ...
> permMode :33204
> ...
>
> which is 40775 in octal, i.e.,
>
> #define S_IFDIR 0040000 /* directory */
>
> To my mind this _is_ a bug. First it's a file catalog - there's no
> reason I can think of that one could usefully enter a dir. Second, a
> replica for a file makes sense. Replicating a directory is clearly
> impossible.
>
> If you try and rep from a DPM you get
>
> ui1-gla:~$ lcg-rf -v --vo dteam srm://se2-gla.scotgrid.ac.uk/dpm/
> scotgrid.ac.uk/home/dteam
> Using grid catalog type: lfc
> Using grid catalog : prod-lfc-shared-central.cern.ch
> set timeout to 0 seconds
> Site URL to be registered: srm://se2-gla.scotgrid.ac.uk/dpm/
> scotgrid.ac.uk/home/dteam
> File size: 0
> Alias created in Catalog: lfn:/grid/dteam/generated/2006-06-16/
> file-4de347c7-f42a-4048-863d-1efd7a4c1d1e
> guid:5d9902d9-33e9-4113-b4cd-b1f017acd045
> ui1-gla:~$ lcg-rep -v --vo dteam -d dpm.epcc.ed.ac.uk srm://se2-
> gla.scotgrid.ac.uk/dpm/scotgrid.ac.uk/home/dteam
> Using grid catalog type: lfc
> Using grid catalog : prod-lfc-shared-central.cern.ch
> lcg_rep: No such file or directory
>
> At least it fails cleanly, even if the error message is just wrong.
>
> I would have thought a savannah bug rather than a GGUS ticket? (CC me
> in.)
>
> Cheers
>
> Graeme
>
>
> On 13 Jun 2006, at 18:02, Steve Traylen wrote:
>
> >I was about to submit this as a GGUS ticket but thought as you are
> >thinking
> >about SRMs today someone may have comment here first.
> >
> >
> >Hi,
> >
> >I found this problem after debugging a script with an atlas user for a
> >couple of hours.
> >While the mistake was in the script the lcg- commands should really
> >be more
> >resillant to this.
> >
> >When registering a SURL in an LcgFC the existance of the SURL is
> >checked with
> >the SRM and quite correctly fails if the SURL does not exist.
> >
> >However it appears that this check is also succesful if the SRM path
> >corresponds to a directory. dCache and Castor handle replications
> >of these
> >LFNs in different ways.
> >
> >1) Register an existing SURL in the LcgFC.
> >
> > For dCache
> >
> > $ lcg-cr -v --vo dteam
> >srm://dcache.gridpp.rl.ac.uk/pnfs/gridpp.rl.ac.uk/data/dteam/
> >existing/file
> >
> >lfn:/grid/dteam/generated/2006-06-13/file-3030ecfd-43fa-46c7-b287-
> >e287c984ef23
> >
> > and for CASTOR
> >
> > $ lcg-cr -v --vo dteam
> >srm://castorsrm.cern.ch/castor/cern.ch/grid/dteam/generated/
> >2006-06-13/file4589136b-d6dd-4b1a-aab3-eeb796ba1b48
> >
> >Alias created in Catalog:
> >lfn:/grid/dteam/generated/2006-06-13/file-
> >e02454c6-149d-444c-805d-0c5f7554b7cd
> >
> >2) Now try registering a non-existing SURL.
> >
> > For CASTOR
> >$ lcg-rf -v --vo dteam
> >srm://castorsrm.cern.ch/castor/cern.ch/grid/dteam/generated/not/
> >existing/file
> >
> >fails quite correctly with
> >
> >lcg_rf: No such file or directory
> >
> >a similar thing can be seen for dcache as well.
> >
> >
> >3) Now try registering an existing directory
> >
> > For dCache
> >
> >$ lcg-rf -v --vo dteam
> >srm://dcache.gridpp.rl.ac.uk/pnfs/gridpp.rl.ac.uk/data/dteam/qmul
> >
> > returns
> >
> >Alias created in Catalog:
> >lfn:/grid/dteam/generated/2006-06-13/
> >file-971a3036-1521-4035-9674-081665f94ea3
> >
> > And for CASTOR
> >
> >$ lcg-rf -v --vo dteam
> >srm://castorsrm.cern.ch/castor/cern.ch/grid/dteam/generated
> >
> >lfn:/grid/dteam/generated/2006-06-13/file-19b4eee3-4f59-480e-
> >ad7f-0f18586ffc04
> >
> >so we have managed to be able to register and LFN and guid for these
> >directories.
> >
> >4) Now try and replicate each of these LFNs to the other server.
> >
> > For castor->dcache
> >
> >$ lcg-rep -v --vo dteam
> >lfn:/grid/dteam/generated/2006-06-13/file-19b4eee3-4f59-480e-
> >ad7f-0f18586ffc04
> >-d dcache.gridpp.rl.ac.uk
> >Using grid catalog type: lfc
> >Using grid catalog : prod-lfc-shared-central.cern.ch
> >
> >this just hangs and has to be killed.
> >
> > For dcache -> castor
> >
> >$ lcg-rep -v --vo dteam
> >lfn:/grid/dteam/generated/2006-06-13/
> >file-971a3036-1521-4035-9674-081665f94ea3
> >-d castorsrm.cern.ch
> >Using grid catalog type: lfc
> >Using grid catalog : prod-lfc-shared-central.cern.ch
> ># streams: 1
> ># set timeout to 0
> > 0 bytes 0.00 KB/sec avg 0.00 KB/sec inst
> >
> > again this just hangs and needs to be killed.
> >
> >5) doing just and lcg-cp
> >
> > for dcache
> >
> >lcg-cp -v --vo dteam
> >lfn:/grid/dteam/generated/2006-06-13/
> >file-971a3036-1521-4035-9674-081665f94ea3
> >file:/etc/group
> >
> ># streams: 1
> ># set timeout to 0 (seconds)
> >a system call failed (Permission denied)
> >lcg_cp: Transport endpoint is not connected
> >
> > which is sensible in that it at least fails.
> >
> > and for castor
> >
> >$ lcg-cp -v --vo dteam
> >lfn:/grid/dteam/generated/2006-06-13/file-19b4eee3-4f59-480e-
> >ad7f-0f18586ffc04
> >file:/etc/junk1
> >Using grid catalog type: lfc
> >Using grid catalog : prod-lfc-shared-central.cern.ch
> >
> > and it just hangs like this.
> >
> >
> >So in conclusion there are a number of points here.
> >
> >1) Should one be able to register a SURL directory with lcg-rf
> > The short answer is probably no you should not be able to do
> > this but it is tricky since SRMs don't know anything about
> >directories
> > and a directory is a file after all. Can one distinguish between
> > a file and directory at the SRM level?
> >
> >2) Both castor and dCache should handle better the case where some
> > one tries to either replicate or copy out a directory. Although
> > the commands above were done with LFNs the same happens when SURLs
> > are used that are actually directory paths.
> >
> > Steve
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >--
> >Steve Traylen
> >[log in to unmask]
> >http://www.gridpp.ac.uk/
>
> --
> Dr Graeme Stewart - http://wiki.gridpp.ac.uk/wiki/User:Graeme_stewart
> GridPP DM Wiki - http://wiki.gridpp.ac.uk/wiki/Data_Management
>
>
>
--
Steve Traylen
[log in to unmask]
http://www.gridpp.ac.uk/
|