FYI, we had some problems with RFIO on our SL5 disk servers at the Tier1
because CASTOR was getting the port for rfiod from /etc/services which in
SL5 includes a definition for RFIO on port 3147 which wasn't present in SL4.
If the entry in /etc/services isn't there then rfiod happily uses port 5001.
Perhaps DPM does something similar?
Just in case...
James.
> -----Original Message-----
> From: GRIDPP2: Deployment and support of SRM and local storage management
> [mailto:[log in to unmask]] On Behalf Of Wahid Bhimji
> Sent: 01 July 2010 12:21
> To: [log in to unmask]
> Subject: Re: RFIO TFile
>
> Hi
>
> Just posting this back to the list in case anyone encounters the same
> problem.
> If you are happy with the outcome then all good. But I don't have 3147
open
> so it is still a bit of a mystery why it is trying to use that port
> (perhaps something you specfied in yaim for those servers 5001 is the
normal
> rfio port (with $RFIO_PORT_RANGE defining the data ports).
>
> Also mystery why athena jobs worked before with rfio local access
(assuming
> they did) which should amount to the same thing.
>
> Anyway as long as it works I guess !
>
> Wahid
>
> On 1 Jul 2010, at 12:10, Chris Curtis wrote:
>
> >
> > Hi -
> >
> > Yes, it looks like a firewall problem. The head node is trying to
> communicate on port 5001, but the pool node is trying port 3147 according
to
> /etc/services. This port is closed according to our iptables definition.
> >
> > Opening 3147 allows root to open the file when I specify either the DPM
or
> physical filename. Mystery solved!
> >
> > Cheers,
> >
> > Chris
> >
> >>>
> >>> On Thu, 1 Jul 2010, Wahid Bhimji wrote:
> >>>
> >>>>
> >>>> Are you doing this from a UI - did you source grid-env.sh.
> >>>>
> >>>> I have used it at ECDF and at Glasgow. The firewall needs to be open
to
> rfio connections from that host but if you can rfcp then that should be
ok.
> >>>>
> >>>> Can you try opening it directly from the disk server. ie like this.
That
> is what I do (you will need to use dpns-to-disk to find the path (or
lcg-gt))
> >>>> TFile *_file0 =
>
TFile::Open("rfio://pool2.glite.ecdf.ed.ac.uk//gridstorage015/atlas/2010-02-
> 15/AODClone.root.5371914.0", "READ");
> >>>>
> >>>> using the dpns path should work but you may need more slashes after
rfio
> something like:
> >>>>
>
TFile::Open("rfio:////epgse1.ph.bham.ac.uk//dpm/ph.bham.ac.uk/home/atlas/atl
a
>
sscratchdisk/user10/jodypalmer/ganga/user10.jodypalmer.ganga.4998.20100628/u
s
>
er10.jodypalmer.ganga.4998.20100628.TopPhys_TopInputs.pt_18.0._00006.root");
> >>>>
> >>>> Cheers
> >>>>
> >>>> Wahid
> >>>>
> >>>>
> >>>> On 1 Jul 2010, at 10:44, Chris Curtis wrote:
> >>>>
> >>>>>
> >>>>> Hi -
> >>>>>
> >>>>> Yes, my proxy is valid. I've tried adding the following to my
script
> (before calling root):
> >>>>>
> >>>>> export DPNS_HOST=epgse1.ph.bham.ac.uk
> >>>>> export DPM_HOST=epgse1.ph.bham.ac.uk
> >>>>>
> >>>>> ln -s /opt/lcg/lib64/libdpm.so libshift.so.2.1
> >>>>> export LD_LIBRARY_PATH=`pwd`:$LD_LIBRARY_PATH
> >>>>>
> >>>>> but this does not seem to help - I still get the same error.
> >>>>>
> >>>>> At which site did you manage to open the TFile? I wonder if I have
> missed something when configuring our nodes?
> >>>>>
> >>>>> Cheers,
> >>>>>
> >>>>> Chris
> >>>>>
> >>>>>
> >>>>> On Thu, 1 Jul 2010, Wahid Bhimji wrote:
> >>>>>
> >>>>>> Hi Chris,
> >>>>>>
> >>>>>> Do you have a proxy (ie. from voms-proxy-init) and do you point to
it
> with
> >>>>>> export X509_USER_PROXY=[pathtoproxy]
> >>>>>>
> >>>>>> You also need the somewhat obscure link of libshift.so.2.1 to
> libdpm.so (and for that to be found in your LD_LIBRARY_PATH)
> >>>>>> (libdpm.so should be in $LCG_LOCATION/lib (or lib64 if you are
using
> 64 bit root).
> >>>>>>
> >>>>>> I have successfully opened a TFile on a DPM SE via RFIO in root so
it
> is definitely possible and your Root syntax for TFile is correct.
> >>>>>>
> >>>>>> Wahid
> >>>>>>
> >>>>>>
> >>>>>> On 30 Jun 2010, at 23:22, Chris Curtis wrote:
> >>>>>>
> >>>>>>> Hi -
> >>>>>>>
> >>>>>>> Apologies for cross posting (original message on ATLAS distributed
> Analysis egroup) - I thought on reflection this question might be more
grid
> oriented.
> >>>>>>>
> >>>>>>> I'm trying to access a Root file on our SE via RFIO. I can use
rfcp
> to download the file, but what I would really like to do is access it from
> within Root by calling:
> >>>>>>>
> >>>>>>> TFile *f =
>
TFile::Open("rfio://epgse1.ph.bham.ac.uk//dpm/ph.bham.ac.uk/home/atlas/atlas
s
>
cratchdisk/user10/jodypalmer/ganga/user10.jodypalmer.ganga.4998.20100628/use
r
> 10.jodypalmer.ganga.4998.20100628.TopPhys_TopInputs.pt_18.0._00006.root");
> >>>>>>>
> >>>>>>> If I do this though root (from either a UI or a WN), root throws
the
> error:
> >>>>>>>
> >>>>>>> SysError in <TRFIOFile::TRFIOFile>: file
> rfio://epgse1.ph.bham.ac.uk//dpm/ph.bham.ac.uk
> >>>>>>>
>
/home/atlas/atlasscratchdisk/user10/jodypalmer/ganga/user10.jodypalmer.ganga
.
> 4998.20100628
> >>>>>>>
>
/user10.jodypalmer.ganga.4998.20100628.TopPhys_TopInputs.pt_18.0._00006.root
> can not be opened for reading (Connection refused)
> >>>>>>>
> >>>>>>> And in the /var/log/rfio/log file on the DPM head node I see the
> corresponding message:
> >>>>>>>
> >>>>>>> rfiod[17490]: Could not establish an authenticated connection:
> Csec_server_negociate_protocol:
> >>>>>>> The client did not send an authentication negociation request;
> _Csec_recv_token: Received magic: 2002001 expecting ca03 !
> >>>>>>>
> >>>>>>> Has anyone seen this before (or successfully opened a TFile on a
DPM
> SE via RFIO in root)?
> >>>>>>>
> >>>>>>> Cheers,
> >>>>>>>
> >>>>>>> Chris
> >>>>>>>
> >>>>>>> --
> >>>>>>> West 326
> >>>>>>> Physics and Astronomy
> >>>>>>> University of Birmingham
> >>>>>>> Edgbaston
> >>>>>>> Birmingham
> >>>>>>> B15 2TT
> >>>>>>>
> >>>>>>> (Office) 0121 414 4700
> >>>>>>> (Mobile) 0798 666 1959
> >>>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>
> >>>>> --
> >>>>> West 326
> >>>>> Physics and Astronomy
> >>>>> University of Birmingham
> >>>>> Edgbaston
> >>>>> Birmingham
> >>>>> B15 2TT
> >>>>>
> >>>>> (Office) 0121 414 4700
> >>>>> (Mobile) 0798 666 1959
> >>>>>
> >>>>
> >>>>
> >>>>
> >>>
> >>> --
> >>> West 326
> >>> Physics and Astronomy
> >>> University of Birmingham
> >>> Edgbaston
> >>> Birmingham
> >>> B15 2TT
> >>>
> >>> (Office) 0121 414 4700
> >>> (Mobile) 0798 666 1959
> >>>
> >>
> >>
> >>
> >
> > --
> > West 326
> > Physics and Astronomy
> > University of Birmingham
> > Edgbaston
> > Birmingham
> > B15 2TT
> >
> > (Office) 0121 414 4700
> > (Mobile) 0798 666 1959
> >
>
>
> --
> The University of Edinburgh is a charitable body, registered in
> Scotland, with registration number SC005336.
|