network tuning

balance-rr TCP re-ordering

#net.ipv4.tcp_reordering = 3 #default
net.ipv4.tcp_reordering = 127

adjusting TCP window size

check the default values first,

sysctl net.core.wmem_max
sysctl net.core.rmem_max
sysctl net.ipv4.tcp_rmem
sysctl net.ipv4.tcp_wmem

sysctl net.ipv4.tcp_reordering

then change those,

vi /etc/sysctl.conf

net.core.wmem_max = 4194304
net.core.rmem_max = 12582912
net.ipv4.tcp_rmem = 4096 87380 4194304
net.ipv4.tcp_wmem = 4096 87380 4194304
#net.ipv4.tcp_reordering = 3
net.ipv4.tcp_reordering = 127

sysctl -p

iperf3 -P3 -c $server -w 8M

jumbo frames

check the current values first,

ifconfig bond0
ifconfig xenbr0
ifconfig eth0
ifconfig eth2
ifconfig eth3
#mtu 1500
#txqueuelen 1000

sysctl net.core.netdev_max_backlog

then edit your init scripts,

ifconfig bond0 mtu 9000 txqueuelen 10000
ifconfig xenbr0 mtu 9000 txqueuelen 10000
ifconfig eth0 mtu 9000 txqueuelen 10000
ifconfig eth2 mtu 9000 txqueuelen 10000
ifconfig eth3 mtu 9000 txqueuelen 10000
#ip link set

vi /etc/sysctl.conf

#net.core.netdev_max_backlog = 1000
net.core.netdev_max_backlog = 3000

sysctl -p

Note. all the NICs on that network should be switched to that MTU.

resources

rr reordering

https://mjmwired.net/kernel/Documentation/networking/bonding.txt#2316

tcp window

https://netbeez.net/blog/tcp-window-size/

https://kb.wisc.edu/uwsysnet/page.php?id=41705

jumbo

https://wiki.archlinux.org/index.php/jumbo_frames

https://serverfault.com/questions/287475/why-does-my-gigabit-bond-not-deliver-at-least-150-mb-s-throughput

https://linuxconfig.org/how-to-enable-jumbo-frames-in-linux

https://unix.stackexchange.com/questions/451647/can-jumbo-frames-mtu-9000-be-set-on-vm-machines