Hi Stephen,
Thanks for all the info.
Burke, S (Stephen) wrote:
>>I then moved on through the LCG-2 User's Guide, which suggested the
>>edg-lrc and edg-lrm commands. These both told me I needed the output
>>from lcg-infosites, so again I was up the proverbial creek.
>
> Well, you need the lhcb LRC and RMC contact URLs:
>
> ldapsearch -x -h lcgbdii02.gridpp.rl.ac.uk -p 2170 -b o=grid | grep -A 1 URL
> | grep -A 1 lhcb
> GlueServiceAccessPointURL:
> http://rlslhcb.cern.ch:7777/lhcb/v2.2/edg-local-rep
> lica-catalog/services/edg-local-replica-catalog
> --
> GlueServiceAccessPointURL:
> http://rlslhcb.cern.ch:7777/lhcb/v2.2/edg-replica-m
> etadata-catalog/services/edg-replica-metadata-catalog
I realised this was the case, but I didn't know what I was looking for.
I'm sure that LDAP query above is as familiar as the back of your hand
... one of my favourite things about LDAP is how intuitive it is to use.
>>This then left me with getting the .BrokerInfo information back from a
>>job which executted at Birmingham, in hopes of finding the path to the
>>local SE within it.
>
> It wouldn't give you anything useful anyway, there is no path any more.
I think there is *some* path, but I don't think it means anything.
>>Interestingly, my first jobs to come back show that epcf38 is
>>mounted on
>>the WNs, and not epcf37 (the listed SE), and furthermore the directory
>>/experiment seems to have the SE files in it, rather than /storage (as
>>listed in .BrokerInfo, and therefore presumably in the IS/MDS/R-GMA).
>
> What you're seeing there is not the SE storage, but the location for storing
> VO software.
That explains a thing or two.
>>In any case, I am now stuck waiting 5-10 minutes for each
>>(effectively)
>>"ls" command to execute and return the results. This turn around time
>>means I have to give up, so my question remains un-answered.
>
>
> You can do it somewhat faster with globus-job-run to the relevant job
> manager, i.e. something like
>
> globus-job-run <ce.host.name>/jobmanager-lcgpbs /bin/ls
>
> which will run the ls as a PBS job.
Thanks for the tip.
>>Of course, I hope that I have just missed something simple.
>>I hope that
>>if the lcg-infosites command had existed then I could have done
>>something like:
>>
>>edg-rmc mappingsByAlias '*lhcb*' --endpoint <birmingham-endpoint>
>
> No, that wouldn't work. There is only one catalogue per VO, hence no
> birmingham endpoint. You would have to use edg-lrc filtering on the SE name
> in the SURL, which would give you a list of GUIDs. If LHCb references files
> by LFN you'd then have to do another query on each guid to get that. The RM
> commands are designed to go from LFN to the SE, but not really vice versa.
So it isn't really possible to see what files a particular VO has on a
particular SE?
>>As I understand it, the LCG data management system uses a
>>flat namespace
>>for file storage, and any "/" (slashes) are just for
>>convenience. There
>>is no (easy) way to list files in a particularly "pseudo-hierarchy"
>>level (files in leaves can be listed using the appropriate
>>wildcard with
>>the "pseudo-path" followed by "/*", I suppose).
>
> I believe that will change with the new LCG file catalogue, but it isn't
> released yet, although some parts are in 2.3.
That is good to know.
>>Also, I have confirmed that /storage does not exist at birmingham, so
>>anyone running jobs there hoping to use the .BrokerInfo file or
>>edg-brokerinfo commands to point them to the mounted local SE storage
>>would be out of luck.
>
> It does exist on the SE, but not on the CE or WNs. LCG has not supported NFS
> mounts since the start of the 2.x releases a year ago. Interestingly, in EDG
> the experiments were always very vociferous in saying that NFS access was an
> absolute requirement, so we put in a lot of effort to try to make it work.
> Since LCG took it away I think hardly anyone has even noticed, and as far as
> I know no-one has complained. So much for requirements!
Could it be that users haven't yet figured out how to properly use the
data management system and therefore haven't realised that they don't
have "regular" NFS access to the close SE? I believe LHCb is only
indirectly using the LCG SEs, and certainly not using much of the Data
Mgmt tools (AFAIK).
And at the end of the day, I still don't feel like I have an answer to
my question "How do I find out what LHCb files are filling up
Birmingham"? Maybe the answer is that we're not, and it is all Atlas.
Even being able to see that would be nice.
Cheers,
Ian.
|