利用openvswitch对广播报文限速
1. 准备环境
CentOS7（1511）
Docker（1.11.2）
OpenVswitch（2.4.0-1）
Iperf（2.0.8-2）
2. 网络拓扑
单宿主机OpenVswitch网络拓扑
+---+
|em1|
+--------+---+-------+
| br-em1 |
+----+----------+----+
|phy-br-int|
+----------+
|
|
+----------+
|int-br-em1|
+---------+----------+-----------+
| br-int |
+--+------+------------+------+--+
|qbi-xx| |qbi-yy|
+------+ +------+
| |
| |
+------+ +------+
|qbv-xx| |qbv-yy|
+--+------+--+ +--+------+--+
| qbr-xx | | qbr-yy |
+--+------+--+ +--+------+--+
|qvb-xx| |qvb-yy|
+------+ +------+
| |
| |
+------+ +------+
|qvo-xx| |qvo-yy|
+--+------+--+ +--+------+--+
| 容器xx | | 容器yy |
+------------+ +------------+
3. 构建网络
3.1 创建容器
docker run -itd --name=magine1 --net=none 0.0.20.56:5000/centos /bin/bash
ddfb7c3015f34f178d7efc112e6d9350b1553c07aaed4164a861c339346dba7e
docker run -itd --name=magine2 --net=none 0.0.20.56:5000/centos /bin/bash
67fe66ff23896010e5590dc8f841a82a957fd5b8694ef2d9b04b316a110854dc
3.2 创建虚拟网卡
容器ddfb7c30
ip link add qbi-ddfb7c30 type veth peer name qbv-ddfb7c30
ip link add qvb-ddfb7c30 type veth peer name qvo-ddfb7c30
容器f52ebaa9
ip link add qbi-f52ebaa9 type veth peer name qbv-f52ebaa9
ip link add pvb-f52ebaa9 type veth peer name qvo-f52ebaa9
3.3 创建ovs拓扑
基础网桥
ovs-vsctl add-br br-int
ovs-vsctl add-br br-em1
ovs-vsctl add-port br-em1 em1
容器ddfb7c30
ovs-vsctl add-br qbr-ddfb7c30
ovs-vsctl add-port br-int qbi-ddfb7c30
ovs-vsctl add-port qbr-ddfb7c30 qbv-ddfb7c30
ovs-vsctl add-port qbr-ddfb7c30 qvb-ddfb7c30
容器f52ebaa9
ovs-vsctl add-br qbr-f52ebaa9
ovs-vsctl add-port br-int qbi-f52ebaa9
ovs-vsctl add-port qbr-f52ebaa9 qbv-f52ebaa9
ovs-vsctl add-port qbr-f52ebaa9 qvb-f52ebaa9
3.4 构建容器NameSpace
容器ddfb7c30
docker inspect -f '{{ .State.Pid }}' ddfb7c3015f3
32687
关联系统namespace路径
ln -s /proc/32687/ns/net /run/netns/ddfb7c3015f3
查看容器ns
ip netns
ddfb7c3015f3
容器分入网卡
ip link set qvo-ddfb7c30 netns ddfb7c3015f3
配置IP
ip netns exec ddfb7c3015f3 ip addr 10.1.1.2/24 dev qvo-ddfb7c30
激活网卡
ip netns exec ddfb7c3015f3 ip link set qvo-ddfb7c30 up
ip link set qvb-ddfb7c30 up
ip link set qbi-ddfb7c30 up
ip link set qbv0ddfb7c30 up
容器f52ebaa9
以上操作
3.5 ovs创建队列和流表
容器ddfb7c30
在qbv-ddfb7c30上创建1条qos，2条队列，其中队列0最高1Gbit，最低500Mbit，队列1最高1Mbit，最低1Mbit
$ ovs-vsctl set port qbv-ddfb7c30 qos=@newqos -- --id=@newqos create qos type=linux-htb other-config:max-rate=1000000000 queues=0=@q0,1=@q1 -- --id=@q0 create queue other-config:min-rate=500000000 other-config:max-rate=1000000000 -- --id=@q1 create queue other-config:min-rate=1000000 other-config:max-rate=1000000
c5c8d139-7443-49f7-81d5-d92c8754e5de
ea91d271-618d-442f-b88c-6ab0bfc377ba
b6f63f0d-193d-4e9d-95c8-2af7e3aac247
查看qbr-ddfb7c30端口信息
$ ovs-ofctl show qbr-ddfb7c30
OFPT_FEATURES_REPLY (xid=0x2): dpid:0000aaee33131e41
n_tables:254, n_buffers:256
capabilities: FLOW_STATS TABLE_STATS PORT_STATS QUEUE_STATS ARP_MATCH_IP
actions: output enqueue set_vlan_vid set_vlan_pcp strip_vlan mod_dl_src mod_dl_dst mod_nw_src mod_nw_dst mod_nw_tos mod_tp_src mod_tp_dst
2(qvb-ddfb7c30): addr:96:60:70:41:70:0c
config: 0
state: 0
current: 10GB-FD COPPER
speed: 10000 Mbps now, 0 Mbps max
3(qbv-ddfb7c30): addr:66:19:ab:4e:7e:09
config: 0
state: 0
current: 10GB-FD COPPER
speed: 10000 Mbps now, 0 Mbps max
LOCAL(qbr-ddfb7c30): addr:aa:ee:33:13:1e:41
config: PORT_DOWN
state: LINK_DOWN
speed: 0 Mbps now, 0 Mbps max
OFPT_GET_CONFIG_REPLY (xid=0x4): frags=normal miss_send_len=0
在qbr-ddfb7c30上创建两条流表，用于匹配广播地址和单播地址
$ ovs-ofctl add-flow qbr-ddfb7c30 "table=1, in_port=2, dl_src=00:00:00:00:00:00/01:00:00:00:00:00, actions=enqueue:3:0"
$ ovs-ofctl add-flow qbr-ddfb7c30 "table=0, in_port=2, dl_src=01:00:00:00:00:00/01:00:00:00:00:00, actions=enqueue:3:1"
查看流表
$ ovs-ofctl dump-flows qbr-ddfb7c30
NXST_FLOW reply (xid=0x4):
cookie=0x0, duration=264250.612s, table=0, n_packets=1465040, n_bytes=31895512642, idle_age=5938, hard_age=65534, priority=0 actions=NORMAL
cookie=0x0, duration=2.382s, table=0, n_packets=0, n_bytes=0, idle_age=2, in_port=2,dl_src=01:00:00:00:00:00/01:00:00:00:00:00 actions=enqueue:3:1
cookie=0x0, duration=11.006s, table=1, n_packets=0, n_bytes=0, idle_age=11, in_port=2,dl_src=00:00:00:00:00:00/01:00:00:00:00:00 actions=enqueue:3:0
容器f52ebaa9
以上操作
4. 测试
由于没有测试广播风暴的工具，就以iperf测试单播为准
4.1 iperf server
宿主机
iperf -s
--------------------------------------
Server listening on TCP port 5001
TCP window size: 85.3 KByte (default)
--------------------------------------
4.2 容器 iperf Client
不配置qos
[root@docker1 ~]# ip netns exec ddfb7c3015f3 iperf -c 10.1.1.1 -i 1 -t 10
------------------------------------------------------------
Client connecting to 10.1.1.1, TCP port 5001
TCP window size: 85.0 KByte (default)
------------------------------------------------------------
[ 3] local 10.1.1.2 port 38143 connected with 10.1.1.1 port 5001
[ ID] Interval Transfer Bandwidth
[ 3] 0.0- 1.0 sec 3.05 GBytes 26.2 Gbits/sec
[ 3] 1.0- 2.0 sec 3.24 GBytes 27.9 Gbits/sec
[ 3] 2.0- 3.0 sec 3.24 GBytes 27.8 Gbits/sec
[ 3] 3.0- 4.0 sec 3.29 GBytes 28.3 Gbits/sec
[ 3] 4.0- 5.0 sec 3.35 GBytes 28.8 Gbits/sec
[ 3] 5.0- 6.0 sec 3.31 GBytes 28.4 Gbits/sec
[ 3] 6.0- 7.0 sec 3.29 GBytes 28.2 Gbits/sec
[ 3] 7.0- 8.0 sec 3.32 GBytes 28.5 Gbits/sec
[ 3] 8.0- 9.0 sec 3.28 GBytes 28.2 Gbits/sec
[ 3] 9.0-10.0 sec 3.26 GBytes 28.0 Gbits/sec
[ 3] 0.0-10.0 sec 32.6 GBytes 28.0 Gbits/sec
配置qos，单播走队列0（正常队列。限速1Gbit）
$ ovs-ofctl add-flow qbr-ddfb7c30 "table=0, in_port=2, dl_src=00:00:00:00:00:00/01:00:00:00:00:00, actions=enqueue:3:0"
[root@docker1 ~]# ip netns exec ddfb7c3015f3 iperf -c 10.1.1.1 -i 1 -t 10
------------------------------------------------------------
Client connecting to 10.1.1.1, TCP port 5001
TCP window size: 85.0 KByte (default)
------------------------------------------------------------
[ 3] local 10.1.1.2 port 38157 connected with 10.1.1.1 port 5001
[ ID] Interval Transfer Bandwidth
[ 3] 0.0- 1.0 sec 117 MBytes 980 Mbits/sec
[ 3] 1.0- 2.0 sec 115 MBytes 965 Mbits/sec
[ 3] 2.0- 3.0 sec 114 MBytes 952 Mbits/sec
[ 3] 3.0- 4.0 sec 114 MBytes 955 Mbits/sec
[ 3] 4.0- 5.0 sec 114 MBytes 955 Mbits/sec
[ 3] 5.0- 6.0 sec 114 MBytes 954 Mbits/sec
[ 3] 6.0- 7.0 sec 114 MBytes 954 Mbits/sec
[ 3] 7.0- 8.0 sec 115 MBytes 967 Mbits/sec
[ 3] 8.0- 9.0 sec 114 MBytes 954 Mbits/sec
[ 3] 9.0-10.0 sec 114 MBytes 955 Mbits/sec
[ 3] 0.0-10.0 sec 1.12 GBytes 958 Mbits/sec
配置qos，单播走队列1（广播队列。限速1Mbit）
$ ovs-ofctl add-flow qbr-ddfb7c30 "table=0, in_port=2, dl_src=00:00:00:00:00:00/01:00:00:00:00:00, actions=enqueue:3:0"
[root@docker1 ~]# ip netns exec ddfb7c3015f3 iperf -c 10.1.1.1 -i 1 -t 10
------------------------------------------------------------
Client connecting to 10.1.1.1, TCP port 5001
TCP window size: 85.0 KByte (default)
------------------------------------------------------------
[ 3] local 10.1.1.2 port 38177 connected with 10.1.1.1 port 5001
[ ID] Interval Transfer Bandwidth
[ 3] 0.0- 1.0 sec 83.4 KBytes 683 Kbits/sec
[ 3] 1.0- 2.0 sec 12.7 KBytes 104 Kbits/sec
[ 3] 2.0- 3.0 sec 276 KBytes 2.26 Mbits/sec
[ 3] 3.0- 4.0 sec 136 KBytes 1.11 Mbits/sec
[ 3] 4.0- 5.0 sec 127 KBytes 1.04 Mbits/sec
[ 3] 5.0- 6.0 sec 191 KBytes 1.56 Mbits/sec
[ 3] 6.0- 7.0 sec 191 KBytes 1.56 Mbits/sec
[ 3] 7.0- 8.0 sec 255 KBytes 2.09 Mbits/sec
[ 3] 8.0- 9.0 sec 382 KBytes 3.13 Mbits/sec
[ 3] 9.0-10.0 sec 382 KBytes 3.13 Mbits/sec
[ 3] 0.0-10.2 sec 1.99 MBytes 1.64 Mbits/sec