 |
 |
On Wed, 2005-10-12 at 21:40 -0700, Rod Walker wrote: > Hi, > Acting on a tip-off I measured the transfer rate from one of the Canadaian > clusters to cern. I discovered that > globus-url-copy -tcp-bs 16777216 .... > is about 25 times faster than with the default buffer size, 2.5MB/s > comapared to 100kB/s. > > Is this indicative of some bad network setting, routing, ..., or is this > to be expected for transatlantic transfers - I mean the size of the > improvement. > > How do I make the LCG utils (lcg-cp) use a larger buffer size? > I found some optimizations from SC2/3 > > net.ipv4.tcp_rmem = 1048576 16777216 33554432 > net.ipv4.tcp_wmem = 1048576 16777216 33554432 > net.ipv4.tcp_mem = 1048576 16777216 33554432 > net.core.rmem_max = 16777215 > net.core.wmem_max = 16777215 > net.core.rmem_default = 4194303 > net.core.wmem_default = 4194303 > > Will these mean that globus-url-copy will no longer need the tcp-bs > argument? This is quite normal for a non-tuned system. For more info read http://www.psc.edu/networking/projects/tcptune/ Setting good [rw]mem values will give you godd long-haul performance without consuming too much memory. You should NOT touch ipv4.tcp_mem!!! EVER!! (unless of course you are a kernel hacker and REALLY know what you are doing) You should also not touch the first value of ipv4.tcp_[rw]mem there is absolutely no need to do that. And there is usually no need to touch core.[rw]mem_default since tcp doesn't care about that one (only udp and other protocols) We use net/core/rmem_max = 83333333 net/core/wmem_max = 83333333 net/ipv4/tcp_rmem = 4096 833333 83333333 net/ipv4/tcp_wmem = 4096 833333 83333333 Also read Documentation/networking/ip-sysctl.txt in the kernel-source to understand exactly what the parameters do.