ubuntu,
apt install iperf3
slackware, with SCTP which gets enabled at compile time without any specific flag, as long as you got the headers available,
sbopkg -i lksctp-tools sbopkg -i iperf3
from source – same thing, about the SCTP headers,
pkill iperf3 ps aux | grep iperf3 removepkg iperf3 wget https://github.com/esnet/iperf/archive/3.6.tar.gz tar xzf 3.6.tar.gz cd iperf-3.6/ ./configure grep -i sctp config.log make -j16 make -j16 install ldconfig
either start the server or select the nearest public one
iperf3 -s
or at startup
echo -n iperf3... /usr/bin/iperf3 -s -D -I /var/run/iperf3.pid && echo done || echo FAIL #/usr/local/bin/iperf3
iperf3 -c SERVER_ADDRESS iperf3 --udp -c SERVER_ADDRESS iperf3 --sctp -c SERVER_ADDRESS iperf3 -P 3 -c SERVER_ADDRESS #1.88 vs 2.68 Gbit/s with balance-rr iperf3 --udp -P 3 -c SERVER_ADDRESS #3.15 Git/s with balance-rr iperf3 --sctp -P 3 -c SERVER_ADDRESS
on the server,
iperf3 -s -p 5101 -D; iperf3 -s -p 5102 -D; iperf3 -s -p 5103 -D
on the client,
server=SERVER_ADDRESS ksh -c "iperf3 -c $server -T s1 -p 5101 &; iperf3 -c $server -T s2 -p 5102 &; iperf3 -c $server -T s3 -p 5103" | egrep 'receiver$|sender$' unset server
#net.ipv4.tcp_reordering = 3 #default net.ipv4.tcp_reordering = 127
check the default values first,
sysctl net.core.wmem_max sysctl net.core.rmem_max sysctl net.ipv4.tcp_rmem sysctl net.ipv4.tcp_wmem sysctl net.ipv4.tcp_reordering
then change those,
vi /etc/sysctl.conf net.core.wmem_max = 4194304 net.core.rmem_max = 12582912 net.ipv4.tcp_rmem = 4096 87380 4194304 net.ipv4.tcp_wmem = 4096 87380 4194304 #net.ipv4.tcp_reordering = 3 net.ipv4.tcp_reordering = 127 sysctl -p iperf3 -P3 -c $server -w 8M
check the current values first,
ifconfig bond0 ifconfig xenbr0 ifconfig eth0 ifconfig eth2 ifconfig eth3 #mtu 1500 #txqueuelen 1000 sysctl net.core.netdev_max_backlog
then edit your init scripts,
ifconfig bond0 mtu 9000 txqueuelen 10000 ifconfig xenbr0 mtu 9000 txqueuelen 10000 ifconfig eth0 mtu 9000 txqueuelen 10000 ifconfig eth2 mtu 9000 txqueuelen 10000 ifconfig eth3 mtu 9000 txqueuelen 10000 #ip link set vi /etc/sysctl.conf #net.core.netdev_max_backlog = 1000 net.core.netdev_max_backlog = 3000 sysctl -p
Note. all the NICs on that network should be switched to that MTU.
Interestingly, when binding the endpoint to their respective interface using -B, LACP returns Retr retries to 0. That looks good. As for balance-rr that does not change anything. In fact, I got more shots with 5x the amount of normal retires when using -B.
balance-rr TCP 3 streams [SUM] 0.00-10.00 sec 2.31 GBytes 1.98 Gbits/sec 2378 sender [SUM] 0.00-10.04 sec 2.30 GBytes 1.97 Gbits/sec receiver balance-rr UDP 3 streams [SUM] 0.00-10.00 sec 3.76 MBytes 3.16 Mbits/sec 0.000 ms 0/441 (0%) sender [SUM] 0.00-10.04 sec 3.76 MBytes 3.14 Mbits/sec 0.037 ms 0/441 (0%) receiver balance-rr SCTP 3 streams [SUM] 0.00-10.00 sec 1.15 GBytes 990 Mbits/sec sender [SUM] 0.00-10.04 sec 1.15 GBytes 986 Mbits/sec receiver balance-rr TCP 3 streams after changing the TCP window size and -w 8M [SUM] 0.00-10.00 sec 3.33 GBytes 2.86 Gbits/sec 3729 sender [SUM] 0.00-10.04 sec 3.32 GBytes 2.84 Gbits/sec receiver