Optimizing shaper — hashing filters (HTB)

I have a very nice shaper in my linux box :-)

How the configurator works — it’s another question, here i will try to describe how one could configure her shaper with hash-based filtering.

So, this page about configuring linux kernel traffic shaper, which uses filter hashing for massive IP processing.

Introduction

In a couple of words::

Shaper, in some sense, is a queueing discipline, which processes queue(s) of packets. Discipline may be classless or classful. Classful means that certain traffic may be shaped by certain class.

For certain traffic to be shaped by certain class, there is certain filter should be configured.

So, in some sense, classful discipline consists of two trees — tree of classes and tree of filters. Filters filter packets to classes, classes shape traffic.

Anyway, these topics discovered in a lot of manuals :-)

Why hashes.

«Filters tree» is a defined sequence of filters, with possible jumps between branches. But if tree has a lot of filters, it takes a lot of time to reach mathing one.

If while filters tree is a sequence of check an IP address for a match, it is far more effective to use hashes. Hash is a table of keys «matching» their «values», so for every key there may be only one value, and hash searching (»which value corresponds to this key?») is very fast.

In our case key is an IP address and value for a key is a filter, which filters a packet to a certain shaping class.

Surely, see also a LARTC HOWTO chapter for this.

Idea

So, if we create a table with 256 cells for /24 network and look for neede cell using IP address as a key, every needed filter we will find by one step. It would be 128 steps (in average) with no hash table.

If we need to manage two /24 networks, we can create three tables, two of them for 256 cells (for /24’s) and one with two cells. First select «proper» large table by «network» key and then find proper filter by IP (host part particularly).

Like this:

Lets take two /24, 192.168.1.0/24 and 192.168.2.0/24.

Third bytes are 00000001 and 00000010 (in binary format) respectively.

So, kernel with the very first filter chooses a cell in two-cells table — by third byte in IP address. In that cell there is a filter, which points to one of two 256-cells table, and uses fourth byte as a key. With this filter kernel can find the last (possibly) filter in particular cell of particular table. (»Posibly» — because that filter can point to elsewhere, be chained to other filter(s)). The last filter filters a packet to a class, which shapes a packet.

The task

We have:

Five networks /24: 192.168.1.0/24, 192.168.2.0/24 .. 192.168.5.0/24
Interface $DEV , directed to these networks

We need:

To shape packets depending on destination IP address
Unclassified traffic should be shaped on a special class

Configuring

We will want to have a script, which will configure our kernel… Or to configure kernel directly:

# it may be useful to distinct between
# configuring and creating script:
#tc="/sbin/tc"
tc="/bin/echo /sbin/tc"

Create root qdisc (class 90 — for unclassified traffic):

$tc qdisc add dev $DEV root handle 1: htb default 90

Root class:

$tc class add dev $DEV parent 1:0 classid 1:1 htb rate 100Mbit

Now we need to create «root filter» — other filters need this:

$tc filter add dev $DEV parent 1:1 prio 10 protocol ip u32

Create a table with five cells, one cell for every /24:

# divisor 5 --- table for 5 cells:
$tc filter add dev $DEV parent 1:1 protocol ip prio 10 handle 8: u32 divisor 5

Now let’s create five tables with 256 cells each:

for i in 1 2 3 4 5; do
    $tc filter add dev $DEV parent 1:1 prio 10 handle ${i}: protocol ip u32 divisor 256
done

Now let’s fill upper table (with 5 cells), it must contain jumps to particular 256-cells table:

for i in 1 2 3 4 5; do
    $tc filter add dev $DEV parent 1:1 protocol ip prio 10 \
        u32 ht 8:$[i-1]: \
        match ip dst 192.168.${i}.0/24 \
        hashkey mask 0x000000ff at 16 \
        link $i:
done

This means: put in cell $[i-1] of table 8 (ht 8:$[i-1]:) a filter, which takes a fourth byte
(hashkey mask 0x000000ff) of destination IP address (match ip dst 192.168.${i}.0/24), and uses it as a key for searching in table $i (link $i:).

Now — a «master filter», which uses tree bits of third byte as a key for searching in a main table:

$tc filter add dev $DEV parent 1:0 protocol ip prio 100 u32 ht 800:: \
    match ip dst 192.168.0.0/21 \
    hashkey mask 0x00000700 at 16 link 8:

It means — root filter (ht 800::) must check destination IP address and, if it matches one our networks, (match ip dst 192.168.0.0/21) use last three bits of third byte of address (hashkey mask 0x00000700 at 16) as a key for searching in table 8 (link 8:).

Now we may create classes/filters for our clients.

This is a very individual process (every administrator decides in which format she will store configuration data — classids, rates etc.), so for the sake of demonstration we will create one clients class and a filter for it:

# class 1: 320 --- parent for clients' classes:
$tc class add dev $DEV parent 1:1 classid 1:230 htb rate 30Mbit ceil 50Mbit quantum 1500 . . .
#
# particular client:
$tc class add dev $DEV parent 1:230 classid 1:431 htb rate 2Mbit ceil 10Mbit quantum 1500 burst . . .
$tc filter add dev $DEV protocol ip parent 1:0 prio 100 u32 ht 3:4: match ip dst 192.168.3.4 flowid 1:431

Last command means — In a fifth cell (numbered from zero!) of third table (for 192.168.3.0/24) out a filter, which will filter packets in class 1:431.

So in a loop we can create classes and filters for every client — and we actually fill our 256-cells tables with these filters.

Real case

In a real case the administrator may want to keep configuration data in a SQL database, real scripts may differ a lot but perform the same job. Such shapers scale quite well, we may double (tripe, quadruple etc ….) number of networks, but filters will work very fast.

Probably you will configure your shaper at a bridge — HTB works quite nice at bridged interfaces, with no IP address assigned.

Notes

When i worked on this configuration, i noticed that it is necessary to create a «whole» number of 256-cells tables: it should be equal to (2^^n – 1), where n is a number of /24 networks. So, i have to create more tables than i actually need, but otherwise it doesn’t work :-)
Cells are numbered starting with zero; cells numbers must be in hex. So, in the example above — u32 ht 3:4: match — four (4) is in hex, too :-)
This is the first beta draft :-)