If you are using srmcp to transfer files between two hosts, one of which
is a dCache, then the dCache will control the transfer and set the number
of parallel streams that are used in any gridftp transfer. Setting
-streams_num on the client has no effect, the server is in control.
I would recommend setting your dCache to using a single parallel stream.
Tests have been done showing that this improves write performance during
FTS transfers since it leads to fewer random writes on the disk. Using
mulitple concurrent _files_ is OK though.
If there are no dCache's involved, then all you can do is srmPut or Get
involving a DPM or CASTOR. With DPM setting -streams_num has no effect.
From the tests that I have run you only ever get one stream. I would
imagine that CASTOR is the same.
On Fri, 7 Jul 2006, Jensen, J (Jens) wrote:
> > One question I have to anyone who knows about srmcp: does it vary the
> > number of gridftp channels by file size? If a small file uses one and a
> > big file more then maybe the latter gets hit by a firewall block and the
> > former doesn't? It might also depend on which of the random globus ports
> > gets picked.
> You should be able to specify the number of streams with the
> -streams_num option, which by default, in the configuration file
> ~/.srmconfig/config.xml, is set to 10.
> AFAIK there is no logic that looks at the file size.
> If I sound catious, it's because I haven't actually experimentally
> verified the number of streams... who knows what actually happens
> without experimental verification...
> >> could not glite-transfer-submit to accept a file://// (why
> >> the 4 slashes?)
> > You should be able to get away with three, or maybe even one, but two
> > probably doesn't work: formally a URL looks like
> > scheme://host/some/path, but for file: you obviously omit the host, and
> > in that case I think the leading // is also optional, but if you say
> > file://tmp/qq it parses tmp as a host name.
> This is an srmcp peccadilloddity.
> Someone (not naming names, *cough*) at FNAL applied this logic: if
> <scheme>:// is the top string, and then the <host> and a slash, then
> surely the filesystem comes after that, i.e. another slash for an
> absolute path. Leaving out the host (= localhost) gives you four
> It's wrong of course; RFC1738 section 3.10 says there shouldn't be a
> top slash on the path (but doesn't specifically say where the top dir
> is). The path is not one to one with Unix file names: /tmp///foo is
> the same as /tmp/foo in Unix, but that would violate the RFC.
Dr Greig A Cowan http://www.ph.ed.ac.uk/~gcowan1
School of Physics, University of Edinburgh, James Clerk Maxwell Building
TIER-2 STORAGE SUPPORT PAGES: http://wiki.gridpp.ac.uk/wiki/Grid_Storage