 |
 |
On 30/06/2010 17:01, Stuart Purdie wrote: > Therefore my conclusions are: > > 1. Unless SACK is disabled on the worker node, SACK is established. > 2. SACK packets are not getting passed the NAT. [1] > 3. This slows down retransmission until the SACK gives up and sends a conventional retransmission ACK. > 4. This makes it all so slow that it hits the timeouts. [2] > > This explains why DPM sites seemed unaffected; YAIM disables SACK on those pool nodes, but not any other SE. [3] > It also explains why bypassing the NAT fixes the problem (but that's not an option in general; too many nodes for the available IP's). Sounds reasonable. Although there's still the Lancaster question, since they're behind NAT and have SACK enabled I believe, and yet have no problems. According to Peter they're running iptables v1.3.5 on an SL5.2 box on their NAT box at Lancaster. At Liverpool we've got iptables v1.4.0 on an Gentoo box. Barring a bug introduced between versions those shouldn't really be that dissimilar, and somehow I suspect the other sites experiencing problems are going to be more likely to have something similar to Lancaster's setup than Liverpool's anyway, so I doubt that's the answer. I wonder if it could be something to do with how INVALID state packets are handled by the iptables config at different sites, e.g. whether they're DROP'd or REJECT'd. When there's more LHCb transfers running I can try some more tests. > Therefore, the advice from me is to set "net.ipv4.tcp_sack = 1" for the worker nodes behind NAT. '= 0' surely? (or am I having an end of day brain-spasm?) -- Robert Fay [log in to unmask] System Administrator office: 220 High Energy Physics Division tel (int): 43396 Oliver Lodge Laboratory tel (ext): +44 (0)151 794 3396 University of Liverpool http://www.liv.ac.uk/physics/hep/