poor man’s bandwidth
This is just a PoC to validate QoS. UDP vs. TCP is just easy to test with iperf3. Ideally I would want to prioritize other things of course.
Queues are active in egress (outbound traffic).
Tagging (CDNR) is done on ingress (inbound traffic).
A queue class name must be unique per interface, but nothing prevents us from using the same name for differents interfaces. This is what we’re doing and also enabling our queue in both directions. We didn’t go that far and play with tags yet, but we could probably also be handling those in both directions.
Be it a XEN or a GENERIC kernel, you need to rebuild your kernel to enable the ALTQ options.
Nevermind the NATs, those are simply used temporarily to install iperf3 on the leaf nodes and then open a few remote shells for running the stress tests.
netbsd92-1
10.0.0.201 # gw 10.0.0.202
10.1.1.254
netbsd92-2
10.0.0.202 # gw 10.0.0.201
10.2.2.254
buster-1
10.1.1.1 # gw 10.1.1.254
192.168.122.201 buster-1
buster-2
10.2.2.1 # gw 10.2.2.254
192.168.122.202 buster-2
enable routing and RED
cp -pi /etc/sysctl.conf /etc/sysctl.conf.dist
vi /etc/sysctl.conf
net.inet.ip.forwarding=1
net.inet6.ip6.forwarding=1
net.inet.tcp.ecn.enable=1
sysctl -w net.inet.ip.forwarding=1
sysctl -w net.inet6.ip6.forwarding=1
sysctl -w net.inet.tcp.ecn.enable=1
check
from buster-1
ping 10.1.1.254
ping 10.0.0.202
ping 10.2.2.1
Choose your protocol number carefully.
ICMP 1 UDP 17 TCP 6
Start with a very low overall bandwidth just for testing.
Here PRIQ for simplicity – from 0 to 15, the latter being the highest priority.
For this purpose, we are giving priority to ICMP and UDP over TCP.
500K max to avoid the 1Mbit/s UDP rate limit on GNS3 (no idea where that one comes from).
vi /etc/altq.conf # new file interface vioif0 bandwidth 500K priq class priq vioif0 class-icmp NULL priority 14 class priq vioif0 class-udp NULL priority 13 class priq vioif0 class-low NULL priority 0 default filter vioif0 class-icmp 0 0 0 0 1 filter vioif0 class-udp 0 0 0 0 0 17 interface vioif1 bandwidth 500K priq class priq vioif1 class-icmp NULL priority 14 class priq vioif1 class-udp NULL priority 13 class priq vioif1 class-low NULL priority 0 default filter vioif1 class-icmp 0 0 0 0 1 filter vioif1 class-udp 0 0 0 0 17 vi /etc/rc.conf altqd=yes /etc/rc.d/altqd start
untested
TOS tags
TOS mask poor man's bandwidth ---------------------------------------------------- Minimum Delay 0x10 VoIP, SSH, WebRTC? Maximum Throughput 0x08 (nothing) Maximum Reliability 0x04 SNMP, DNS Minimum Cost 0x02 NNTP, SMTP ? 0x60
ideally we would want SNMP and DNS to be both lowdelay AND reliable.
Tagging the datagrams is done as follows (inbound traffic)
conditioner vioif0 cdnr-voip <mark 0x10>
untested
Now if you tag the packets from another router using a conditioner, one would go as follows to further classify the network stream.
filter vioif0 class-voip 0 0 0 0 0 tos 0x10
ALTQ provides some statistics by itself
altqstat -i vioif0 altqstat -i vioif1
TCP first
buster-1
iperf3 --time 0 -c 10.2.2.1
meanwhile UDP
iperf3 --time 0 -u -c 10.2.2.1 -p 5202
then ICMP
ping -f 10.2.2.1
You might need to change the UDP bandwidth manually, see the iperf3 manual. Also notice there are two more columns with the UDP output of iperf3.
Sniff
tcpdump -i NETIF host ENDPOINT -vvv
Alternatives to iPerf3
pkgsrc -- net/netperf pkgsrc -- net/aria2
QoS https://nethence.com/lectures/network07-qos.slides.pdf
QoS Resources https://nethence.com/lectures/network07-qos.resources.pdf
Public iPerf3 servers https://iperf.fr/iperf-servers.php
Public Iperf3 servers https://iperf.cc/
Protocol Numbers https://www.iana.org/assignments/protocol-numbers/protocol-numbers.xhtml
altqd traffic shaping example https://wiki.netbsd.org/tutorials/altqd_traffic_shaping_example/
altq.conf – ALTQ configuration file https://man.netbsd.org/altq.conf.5
ALTQ: Alternate Queueing for BSD UNIX (version 3.1) https://www.iijlab.net/~kjc/software/ https://www2.sonycsl.co.jp/person/kjc/kjc/software.html
ALTQ Tips https://www.iijlab.net/~kjc/software/TIPS.txt https://www2.sonycsl.co.jp/person/kjc/software/TIPS.txt
(deprecated –> NPF) https://www.netbsd.org/docs/network/pf.html#altq
src/usr.sbin/altq/altqd/altq.conf.samples/ http://cvsweb.netbsd.org/bsdweb.cgi/src/usr.sbin/altq/altqd/altq.conf.samples/?only_with_tag=MAIN
ping – send ICMP ECHO_REQUEST packets to network hosts https://man.netbsd.org/ping.8 –> Flood
Change between iPerf 2.0, iPerf 3.0 and iPerf 3.1 https://iperf.fr/iperf-doc.php#3change –> Setting DSCP/TOS bit vectors (-S)
iPerf 3 user documentation https://iperf.fr/iperf-doc.php#3doc –> -S, –tos n
benchmarks/netperf - The NetBSD Packages Collection http://ftp.netbsd.org/pub/pkgsrc/current/pkgsrc/benchmarks/netperf/README.html
net/aria2 - The NetBSD Packages Collection http://ftp.netbsd.org/pub/pkgsrc/current/pkgsrc/net/aria2/README.html
List of IP protocol numbers https://en.wikipedia.org/wiki/List_of_IP_protocol_numbers
Implementation of Quality of Service (QoS) with Dynamic Resource Allocation https://www.lrrb.org/pdf/200434.pdf
ALTQ Tips https://www.iijlab.net/~kjc/software/TIPS.txt