Hi All, Recently we observe slow WAN transfer rate from 10Gb server to outside. It is unstable and slow when we run iperf test. We confirm network is fine because no problem in 1Gb interface. Any place we need to tune for long distance data transfer (rtt = ~290ms) through 10Gb interface? Here are current values we give for system parameters and 10Gb network device: # ifconfig eth2 Link encap:Ethernet HWaddr 00:05:1E:C5:8E:A9 inet addr:117.103.102.170 Bcast:117.103.103.255 Mask:255.255.252.0 inet6 addr: fe80::205:1eff:fec5:8ea9/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:9000 Metric:1 RX packets:1863703850 errors:0 dropped:0 overruns:48525 frame:0 TX packets:898956240 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:10000 RX bytes:2151106910237 (1.9 TiB) TX bytes:4993390925930 (4.5 TiB) Memory:95a40000-95a7ffff # increase default and maximum Linux TCP buffer sizes net.ipv4.conf.all.arp_filter = 1 net.ipv4.conf.all.rp_filter = 0 net.ipv4.ipfrag_low_thresh = 524288 net.ipv4.ipfrag_high_thresh = 589824 net.ipv4.ip_no_pmtu_disc = 0 net.ipv4.tcp_syncookies = 1 net.ipv4.tcp_max_syn_backlog = 4096 net.ipv4.tcp_window_scaling = 1 net.ipv4.tcp_sack = 1 net.ipv4.tcp_timestamps = 0 net.ipv4.tcp_rfc1337 = 1 net.ipv4.tcp_adv_win_scale = 2 net.ipv4.tcp_rmem = 65536 87380 16777216 net.ipv4.tcp_wmem = 65536 87380 16777216 net.ipv4.tcp_mem = 1310720 2621440 3145728 # increase Linux TCP buffer limits net.core.rmem_default = 2097152 net.core.wmem_default = 2097152 net.core.rmem_max = 536870912 net.core.wmem_max = 536870912 net.core.optmem_max = 536870912 net.ipv4.tcp_keepalive_intvl = 30 net.ipv4.tcp_keepalive_probes = 5 net.ipv4.tcp_keepalive_time = 7200 net.core.netdev_max_backlog = 250000 # ethtool -c eth2 Coalesce parameters for eth2: Adaptive RX: on TX: off stats-block-usecs: 0 sample-interval: 0 pkt-rate-low: 0 pkt-rate-high: 0 rx-usecs: 180 rx-frames: 0 rx-usecs-irq: 0 rx-frames-irq: 0 tx-usecs: 100 tx-frames: 32 tx-usecs-irq: 0 tx-frames-irq: 0 rx-usecs-low: 0 rx-frame-low: 0 tx-usecs-low: 0 tx-frame-low: 0 rx-usecs-high: 0 rx-frame-high: 0 tx-usecs-high: 0 tx-frame-high: 0 # ethtool -g eth2 Ring parameters for eth2: Pre-set maximums: RX: 32768 RX Mini: 0 RX Jumbo: 0 TX: 65536 Current hardware settings: RX: 2048 RX Mini: 0 RX Jumbo: 0 TX: 2048 # ethtool -k eth2 Offload parameters for eth2: Cannot get device udp large send offload settings: Operation not supported rx-checksumming: on tx-checksumming: on scatter-gather: on tcp segmentation offload: on udp fragmentation offload: off generic segmentation offload: off generic-receive-offload: on Thanks a lot. Regards, Jhen-Wei -- OPS Team, ASGC, Taiwan Tel: +886-2-2789-8311 Fax: +886-2-2783-7653