On Debian Buster, Kernel 5.4.51, I have two interfaces
tap1 joined in a bond interface in mode
balance-xor to increase throughput. There is, however, some traffic that must be sent through
tap0. For the rest I don’t care.
Theoretically, the bond driver can do that using
tc filters and multiq, as documented in the driver docs. I can see in the statistics that the queues are claimed to be used, but inspecting the traffic on the two interfaces shows that the filter is not respected.
Here’s what I did:
I assigned each of the tap interfaces to a queue on the bond, set the queuing discipline to multiqueue and then used
tc to override the queuing decision of the bond to force traffic to
192.168.1.100 (as an example) to always take
# echo "tap0:1" > /sys/class/net/bond0/bonding/queue_id # echo "tap1:2" > /sys/class/net/bond0/bonding/queue_id # tc qdisc add dev bond0 handle 1 root multiq # tc filter add dev bond0 protocol ip parent 1: prio 1 u32 match ip dst 192.168.1.100 action skbedit queue_mapping 1
tc stats, you can see that the different queues are actually used:
# tc -s class show dev bond0 class multiq 1:1 parent 1: Sent 377256252 bytes 2526104 pkt (dropped 0, overlimits 0 requeues 0) backlog 0b 0p requeues 0 class multiq 1:2 parent 1: Sent 21031 bytes 2982 pkt (dropped 0, overlimits 0 requeues 0) backlog 0b 0p requeues 0 class multiq 1:3 parent 1: Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) backlog 0b 0p requeues 0
Most traffic takes the generic queue, the special traffic takes the first of the two interface-specific queues. If I delete the
tc filter again, then the packet counter on the specific queue 1:2 stops.
(Note that the queue numbers between the bonding driver and tc are offset by 1, so the queue 1:1 means “let the driver decide”, queue 1:2 means “always go through
tap0“, queue 1:3 means “always go through
The queues are also mapped to the interfaces:
# cat /proc/net/bonding/bond Ethernet Channel Bonding Driver: v3.7.1 (April 27, 2011) Bonding Mode: load balancing (xor) Transmit Hash Policy: layer3+4 (1) MII Status: up MII Polling Interval (ms): 1000 Up Delay (ms): 0 Down Delay (ms): 0 Peer Notification Delay (ms): 0 Slave Interface: tap0 MII Status: up Speed: 10 Mbps Duplex: full Link Failure Count: 0 Permanent HW addr: xx:xx:xx:xx:xx:89 Slave queue ID: 1 Slave Interface: tap1 MII Status: up Speed: 10 Mbps Duplex: full Link Failure Count: 0 Permanent HW addr: xx:xx:xx:xx:xx:d6 Slave queue ID: 2
tcpdump the two taps on the receiving end individually though, I can clearly see that no matter which queue is used, the special traffic actually still takes either interface using the
balance-xor rule. Now – where did I miss something?