Print

Print
Hi All,

Recently we observe slow WAN transfer rate from 10Gb server to outside.
It is unstable and slow when we run iperf test.
We confirm network is fine because no problem in 1Gb interface.
Any place we need to tune for long distance data transfer (rtt = ~290ms) through 10Gb interface?

Here are current values we give for system parameters and 10Gb network device:
# ifconfig
eth2      Link encap:Ethernet  HWaddr 00:05:1E:C5:8E:A9
          inet addr:117.103.102.170  Bcast:117.103.103.255  Mask:255.255.252.0
          inet6 addr: fe80::205:1eff:fec5:8ea9/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:9000  Metric:1
          RX packets:1863703850 errors:0 dropped:0 overruns:48525 frame:0
          TX packets:898956240 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:10000
          RX bytes:2151106910237 (1.9 TiB)  TX bytes:4993390925930 (4.5 TiB)
          Memory:95a40000-95a7ffff


# increase default and maximum Linux TCP buffer sizes
net.ipv4.conf.all.arp_filter = 1
net.ipv4.conf.all.rp_filter = 0
net.ipv4.ipfrag_low_thresh = 524288
net.ipv4.ipfrag_high_thresh = 589824
net.ipv4.ip_no_pmtu_disc = 0
net.ipv4.tcp_syncookies = 1
net.ipv4.tcp_max_syn_backlog = 4096
net.ipv4.tcp_window_scaling = 1
net.ipv4.tcp_sack = 1
net.ipv4.tcp_timestamps = 0
net.ipv4.tcp_rfc1337 = 1
net.ipv4.tcp_adv_win_scale = 2
net.ipv4.tcp_rmem = 65536 87380 16777216
net.ipv4.tcp_wmem = 65536 87380 16777216
net.ipv4.tcp_mem = 1310720 2621440 3145728

# increase Linux TCP buffer limits
net.core.rmem_default = 2097152
net.core.wmem_default = 2097152
net.core.rmem_max = 536870912
net.core.wmem_max = 536870912
net.core.optmem_max = 536870912
net.ipv4.tcp_keepalive_intvl = 30
net.ipv4.tcp_keepalive_probes = 5
net.ipv4.tcp_keepalive_time = 7200
net.core.netdev_max_backlog = 250000


# ethtool -c eth2
Coalesce parameters for eth2:
Adaptive RX: on  TX: off
stats-block-usecs: 0
sample-interval: 0
pkt-rate-low: 0
pkt-rate-high: 0

rx-usecs: 180
rx-frames: 0
rx-usecs-irq: 0
rx-frames-irq: 0

tx-usecs: 100
tx-frames: 32
tx-usecs-irq: 0
tx-frames-irq: 0

rx-usecs-low: 0
rx-frame-low: 0
tx-usecs-low: 0
tx-frame-low: 0

rx-usecs-high: 0
rx-frame-high: 0
tx-usecs-high: 0
tx-frame-high: 0


# ethtool -g eth2
Ring parameters for eth2:
Pre-set maximums:
RX:             32768
RX Mini:        0
RX Jumbo:       0
TX:             65536
Current hardware settings:
RX:             2048
RX Mini:        0
RX Jumbo:       0
TX:             2048


# ethtool -k eth2
Offload parameters for eth2:
Cannot get device udp large send offload settings: Operation not supported
rx-checksumming: on
tx-checksumming: on
scatter-gather: on
tcp segmentation offload: on
udp fragmentation offload: off
generic segmentation offload: off
generic-receive-offload: on



Thanks a lot.

Regards,
Jhen-Wei


--
OPS Team, ASGC, Taiwan
Tel:  +886-2-2789-8311
Fax: +886-2-2783-7653