0. 关于ovs VLAN
VLAN(Virtual Local Area Network)即虚拟局域网, 按照功能, 部门等因素将网络中的机器进行划分, 使之分属不同的部分, 每一个部分形成一个虚拟的局域网络, 共享一个单独的广域网. 将一个大型交换网络划分为许多个独立的广域网, 即VLAN.
目前OVS的端口支持四种VLAN模式,分别是trunk、access、native-tagged、native-untagged,刚创建的端口默认是不设置任何的VLAN模式。
模式 | 说明 |
---|---|
默认 | 在默认模式下(VLAN_mode没被设置),如果指定了端口的tag属性,那么这个端口就工作在access模式,并且其trunk属性的值应该保持为空。否则,这个port就工作在trunk模式下,如果trunk被指定,则使用指定的trunk值。 |
trunk | trunk模式的端口允许传输所有在其trunk属性中指定的那些VLAN对应的数据包。其他VLAN的数据包就会被丢弃。从trunk模式的端口中进入的数据包其VLAN ID不会发生变化。如果进入的数据包不含有VLAN ID,则该数据包进入交换机后的VLAN为0。从trunk模式的端口出去的数据包,如果VLAN ID不为空,则依然保持该VLAN ID,如果VLAN ID为空,则出去后不再包含802.1Q头部 |
access | access模式的端口只允许不带VLAN的数据包进入,不管数据包的VLAN ID是否与其tag相同,只要含有VLAN ID,这个数据包都会被端口drop。数据包进入access端口后会被打上和端口tag相同的VLAN,而再从access端口出去时,数据包的VLAN会被删除,也就是说从access的端口出去的数据包和进来时一样是不带VLAN的。 |
来自于https://zpzhou.com/archives/openvswitch_vlan.html
当PC1向PC3发送数据时, PC1将IP包封装在以太帧中, 帧的目的地址为PC3的地址, 此时帧中并没有tag信息. 当帧到达Port1时, Port1给帧打上tag(VID=100), 帧进入switch1, 然后帧通过Port3,Port4到达Switch2(假设Port3,Port4的trunks=[100,200], 所以Port3,Port4允许VLAN ID为100,200的帧通过). 在Switch2中, Port5所标记的VID和帧相同, MAC地址也匹配,帧就发送到Port5上, Port5将帧的tag信息去掉, 然后发送给PC3. 由于PC2,PC4与PC1的VLAN不同, 因此收不到PC1发出的帧.
此部分来自于容器与容器云
1. 前言
本文将会利用ovs-docker,(关于ovs-docker可以参考[docker 网络] ovs-docker 使用及原理) 在同一台机器启动四个容器. 然后为容器加valn tag来进行容器隔离.
1.1 docker版本
[root@vm1 ~]# docker version
Client:
Version: 18.09.6
API version: 1.39
Go version: go1.10.8
Git commit: 481bc77156
Built: Sat May 4 02:34:58 2019
OS/Arch: linux/amd64
Experimental: false
Server: Docker Engine - Community
Engine:
Version: 18.09.6
API version: 1.39 (minimum version 1.12)
Go version: go1.10.8
Git commit: 481bc77
Built: Sat May 4 02:02:43 2019
OS/Arch: linux/amd64
Experimental: false
[root@vm1 ~]# ovs-vsctl add-br br0
[root@vm1 ~]# ifconfig br0 192.168.0.250/24
[root@vm1 ~]# ovs-vsctl show
c152c245-2f6c-478c-9c07-2e4a3c7a2403
Bridge "br0"
Port "br0"
Interface "br0"
type: internal
ovs_version: "2.5.1"
2. 配置
创建四个容器
[root@vm1 ~]# docker run -d --name con1 --net=none --privileged=true busybox top
30bdd285fc47e047e879a83f4a36a0c303ad6a38462da464b11b6266387684cd
[root@vm1 ~]# docker run -d --name con2 --net=none --privileged=true busybox top
eb0872370afd0ac1e544ae99522c4aff76abf0716283e4a41c6da471cb82ba21
[root@vm1 ~]# docker run -d --name con3 --net=none --privileged=true busybox top
db1a0b70e84e24d616d4f01c9518312794e12477779a4f8e2405427d754b0f94
[root@vm1 ~]# docker run -d --name con4 --net=none --privileged=true busybox top
247ef66e918147694f930db9e5f92afb8d2b4654b6f9959fb25ef5253bf9c887
利用ovs-docker给这些容器配置网络, 关于ovs-docker可以参考 [docker 网络] ovs-docker 使用及原理
[root@vm1 ~]# ovs-docker add-port br0 eth0 con1 --ipaddress=192.168.0.1/24 --gateway=192.168.0.250
[root@vm1 ~]# ovs-docker add-port br0 eth0 con2 --ipaddress=192.168.0.2/24 --gateway=192.168.0.250
[root@vm1 ~]# ovs-docker add-port br0 eth0 con3 --ipaddress=192.168.0.3/24 --gateway=192.168.0.250
[root@vm1 ~]# ovs-docker add-port br0 eth0 con4 --ipaddress=192.168.0.4/24 --gateway=192.168.0.250
[root@vm1 ~]# ovs-vsctl show
c152c245-2f6c-478c-9c07-2e4a3c7a2403
Bridge "br0"
Port "afc212d089e44_l"
Interface "afc212d089e44_l"
Port "1a18677188d94_l"
Interface "1a18677188d94_l"
Port "07bfe3100ddd4_l"
Interface "07bfe3100ddd4_l"
Port "br0"
Interface "br0"
type: internal
Port "6bf90751c96c4_l"
Interface "6bf90751c96c4_l"
ovs_version: "2.5.1"
[root@vm1 ~]#
2.1 测试
测试 : 从con1可以访问br0, con1, con2, con3, con4.
[root@vm1 ~]# docker exec -it con1 ping -c 1 192.168.0.250
PING 192.168.0.250 (192.168.0.250): 56 data bytes
64 bytes from 192.168.0.250: seq=0 ttl=64 time=5.501 ms
--- 192.168.0.250 ping statistics ---
1 packets transmitted, 1 packets received, 0% packet loss
round-trip min/avg/max = 5.501/5.501/5.501 ms
[root@vm1 ~]# docker exec -it con1 ping -c 1 192.168.0.1
PING 192.168.0.1 (192.168.0.1): 56 data bytes
64 bytes from 192.168.0.1: seq=0 ttl=64 time=0.053 ms
--- 192.168.0.1 ping statistics ---
1 packets transmitted, 1 packets received, 0% packet loss
round-trip min/avg/max = 0.053/0.053/0.053 ms
[root@vm1 ~]# docker exec -it con1 ping -c 1 192.168.0.2
PING 192.168.0.2 (192.168.0.2): 56 data bytes
64 bytes from 192.168.0.2: seq=0 ttl=64 time=4.844 ms
--- 192.168.0.2 ping statistics ---
1 packets transmitted, 1 packets received, 0% packet loss
round-trip min/avg/max = 4.844/4.844/4.844 ms
[root@vm1 ~]# docker exec -it con1 ping -c 1 192.168.0.3
PING 192.168.0.3 (192.168.0.3): 56 data bytes
64 bytes from 192.168.0.3: seq=0 ttl=64 time=5.708 ms
--- 192.168.0.3 ping statistics ---
1 packets transmitted, 1 packets received, 0% packet loss
round-trip min/avg/max = 5.708/5.708/5.708 ms
[root@vm1 ~]# docker exec -it con1 ping -c 1 192.168.0.4
PING 192.168.0.4 (192.168.0.4): 56 data bytes
64 bytes from 192.168.0.4: seq=0 ttl=64 time=4.738 ms
--- 192.168.0.4 ping statistics ---
1 packets transmitted, 1 packets received, 0% packet loss
round-trip min/avg/max = 4.738/4.738/4.738 ms
[root@vm1 ~]#
别的容器也可以访问con1.
[root@vm1 ~]# docker exec -it con2 ping -c 1 192.168.0.1
PING 192.168.0.1 (192.168.0.1): 56 data bytes
64 bytes from 192.168.0.1: seq=0 ttl=64 time=4.524 ms
--- 192.168.0.1 ping statistics ---
1 packets transmitted, 1 packets received, 0% packet loss
round-trip min/avg/max = 4.524/4.524/4.524 ms
[root@vm1 ~]# docker exec -it con3 ping -c 1 192.168.0.1
PING 192.168.0.1 (192.168.0.1): 56 data bytes
64 bytes from 192.168.0.1: seq=0 ttl=64 time=5.441 ms
--- 192.168.0.1 ping statistics ---
1 packets transmitted, 1 packets received, 0% packet loss
round-trip min/avg/max = 5.441/5.441/5.441 ms
[root@vm1 ~]# docker exec -it con4 ping -c 1 192.168.0.1
PING 192.168.0.1 (192.168.0.1): 56 data bytes
64 bytes from 192.168.0.1: seq=0 ttl=64 time=4.805 ms
--- 192.168.0.1 ping statistics ---
1 packets transmitted, 1 packets received, 0% packet loss
round-trip min/avg/max = 4.805/4.805/4.805 ms
[root@vm1 ~]#
3 加tag
为容器打上tag
[root@vm1 ~]# ovs-vsctl show
c152c245-2f6c-478c-9c07-2e4a3c7a2403
Bridge "br0"
Port "afc212d089e44_l"
Interface "afc212d089e44_l"
Port "1a18677188d94_l"
Interface "1a18677188d94_l"
Port "07bfe3100ddd4_l"
Interface "07bfe3100ddd4_l"
Port "br0"
Interface "br0"
type: internal
Port "6bf90751c96c4_l"
Interface "6bf90751c96c4_l"
ovs_version: "2.5.1"
[root@vm1 ~]# ovs-vsctl list interface afc212d089e44_l | grep container_id
external_ids : {container_id="con4", container_iface="eth0"}
[root@vm1 ~]# ovs-vsctl set port afc212d089e44_l tag=200
[root@vm1 ~]# ovs-vsctl list interface 1a18677188d94_l | grep container_id
external_ids : {container_id="con2", container_iface="eth0"}
[root@vm1 ~]# ovs-vsctl set port 1a18677188d94_l tag=100
[root@vm1 ~]# ovs-vsctl list interface 07bfe3100ddd4_l | grep container_id
external_ids : {container_id="con3", container_iface="eth0"}
[root@vm1 ~]# ovs-vsctl set port 07bfe3100ddd4_l tag=200
[root@vm1 ~]# ovs-vsctl list interface 6bf90751c96c4_l | grep container_id
external_ids : {container_id="con1", container_iface="eth0"}
[root@vm1 ~]# ovs-vsctl set port 6bf90751c96c4_l tag=100
[root@vm1 ~]# ovs-vsctl show
c152c245-2f6c-478c-9c07-2e4a3c7a2403
Bridge "br0"
Port "afc212d089e44_l"
tag: 200
Interface "afc212d089e44_l"
Port "1a18677188d94_l"
tag: 100
Interface "1a18677188d94_l"
Port "07bfe3100ddd4_l"
tag: 200
Interface "07bfe3100ddd4_l"
Port "br0"
Interface "br0"
type: internal
Port "6bf90751c96c4_l"
tag: 100
Interface "6bf90751c96c4_l"
ovs_version: "2.5.1"
3.1 测试
测试容器互通性
con1可以访问con2, 但是访问不了con3和con4.
[root@vm1 ~]# docker exec -it con1 ping -c 1 192.168.0.1
PING 192.168.0.1 (192.168.0.1): 56 data bytes
64 bytes from 192.168.0.1: seq=0 ttl=64 time=0.047 ms
--- 192.168.0.1 ping statistics ---
1 packets transmitted, 1 packets received, 0% packet loss
round-trip min/avg/max = 0.047/0.047/0.047 ms
[root@vm1 ~]# docker exec -it con1 ping -c 1 192.168.0.2
PING 192.168.0.2 (192.168.0.2): 56 data bytes
64 bytes from 192.168.0.2: seq=0 ttl=64 time=5.429 ms
--- 192.168.0.2 ping statistics ---
1 packets transmitted, 1 packets received, 0% packet loss
round-trip min/avg/max = 5.429/5.429/5.429 ms
[root@vm1 ~]#
[root@vm1 ~]# docker exec -it con1 ping -c 1 192.168.0.3
PING 192.168.0.3 (192.168.0.3): 56 data bytes
--- 192.168.0.3 ping statistics ---
1 packets transmitted, 0 packets received, 100% packet loss
[root@vm1 ~]# docker exec -it con1 ping -c 1 192.168.0.4
PING 192.168.0.4 (192.168.0.4): 56 data bytes
--- 192.168.0.4 ping statistics ---
1 packets transmitted, 0 packets received, 100% packet loss
con2 可以访问con1, 但是con2 不能访问con3和con4
[root@vm1 ~]# docker exec -it con2 ping -c 1 192.168.0.2
PING 192.168.0.2 (192.168.0.2): 56 data bytes
64 bytes from 192.168.0.2: seq=0 ttl=64 time=0.062 ms
--- 192.168.0.2 ping statistics ---
1 packets transmitted, 1 packets received, 0% packet loss
round-trip min/avg/max = 0.062/0.062/0.062 ms
[root@vm1 ~]# docker exec -it con2 ping -c 1 192.168.0.1
PING 192.168.0.1 (192.168.0.1): 56 data bytes
64 bytes from 192.168.0.1: seq=0 ttl=64 time=3.548 ms
--- 192.168.0.1 ping statistics ---
1 packets transmitted, 1 packets received, 0% packet loss
round-trip min/avg/max = 3.548/3.548/3.548 ms
[root@vm1 ~]# docker exec -it con2 ping -c 1 192.168.0.3
PING 192.168.0.3 (192.168.0.3): 56 data bytes
--- 192.168.0.3 ping statistics ---
1 packets transmitted, 0 packets received, 100% packet loss
[root@vm1 ~]# docker exec -it con2 ping -c 1 192.168.0.4
PING 192.168.0.4 (192.168.0.4): 56 data bytes
--- 192.168.0.4 ping statistics ---
1 packets transmitted, 0 packets received, 100% packet loss
[root@vm1 ~]#
con3不能访问con1和con2, 但是可以访问con4
[root@vm1 ~]# docker exec -it con3 ping -c 1 192.168.0.1
PING 192.168.0.1 (192.168.0.1): 56 data bytes
--- 192.168.0.1 ping statistics ---
1 packets transmitted, 0 packets received, 100% packet loss
[root@vm1 ~]# docker exec -it con3 ping -c 1 192.168.0.2
PING 192.168.0.2 (192.168.0.2): 56 data bytes
--- 192.168.0.2 ping statistics ---
1 packets transmitted, 0 packets received, 100% packet loss
[root@vm1 ~]# docker exec -it con3 ping -c 1 192.168.0.3
PING 192.168.0.3 (192.168.0.3): 56 data bytes
64 bytes from 192.168.0.3: seq=0 ttl=64 time=0.054 ms
--- 192.168.0.3 ping statistics ---
1 packets transmitted, 1 packets received, 0% packet loss
round-trip min/avg/max = 0.054/0.054/0.054 ms
[root@vm1 ~]# docker exec -it con3 ping -c 1 192.168.0.4
PING 192.168.0.4 (192.168.0.4): 56 data bytes
64 bytes from 192.168.0.4: seq=0 ttl=64 time=3.399 ms
--- 192.168.0.4 ping statistics ---
1 packets transmitted, 1 packets received, 0% packet loss
round-trip min/avg/max = 3.399/3.399/3.399 ms
[root@vm1 ~]#
con4 不能访问 con1 和 con2, 但是可以访问con3.
[root@vm1 ~]# docker exec -it con4 ping -c 1 192.168.0.1
PING 192.168.0.1 (192.168.0.1): 56 data bytes
--- 192.168.0.1 ping statistics ---
1 packets transmitted, 0 packets received, 100% packet loss
[root@vm1 ~]# docker exec -it con4 ping -c 1 192.168.0.2
PING 192.168.0.2 (192.168.0.2): 56 data bytes
--- 192.168.0.2 ping statistics ---
1 packets transmitted, 0 packets received, 100% packet loss
[root@vm1 ~]# docker exec -it con4 ping -c 1 192.168.0.3
PING 192.168.0.3 (192.168.0.3): 56 data bytes
64 bytes from 192.168.0.3: seq=0 ttl=64 time=5.234 ms
--- 192.168.0.3 ping statistics ---
1 packets transmitted, 1 packets received, 0% packet loss
round-trip min/avg/max = 5.234/5.234/5.234 ms
[root@vm1 ~]# docker exec -it con4 ping -c 1 192.168.0.4
PING 192.168.0.4 (192.168.0.4): 56 data bytes
64 bytes from 192.168.0.4: seq=0 ttl=64 time=0.065 ms
--- 192.168.0.4 ping statistics ---
1 packets transmitted, 1 packets received, 0% packet loss
round-trip min/avg/max = 0.065/0.065/0.065 ms
[root@vm1 ~]#
3.2 访问br0
访问br0, con1,con2,con3,con4均不能访问br0
[root@vm1 ~]# docker exec -it con1 ping -c 1 192.168.0.250
PING 192.168.0.250 (192.168.0.250): 56 data bytes
--- 192.168.0.250 ping statistics ---
1 packets transmitted, 0 packets received, 100% packet loss
[root@vm1 ~]#
[root@vm1 ~]# docker exec -it con2 ping -c 1 192.168.0.250
PING 192.168.0.250 (192.168.0.250): 56 data bytes
--- 192.168.0.250 ping statistics ---
1 packets transmitted, 0 packets received, 100% packet loss
[root@vm1 ~]# docker exec -it con3 ping -c 1 192.168.0.250
PING 192.168.0.250 (192.168.0.250): 56 data bytes
--- 192.168.0.250 ping statistics ---
1 packets transmitted, 0 packets received, 100% packet loss
[root@vm1 ~]# docker exec -it con4 ping -c 1 192.168.0.250
PING 192.168.0.250 (192.168.0.250): 56 data bytes
--- 192.168.0.250 ping statistics ---
1 packets transmitted, 0 packets received, 100% packet loss
为br0设置tag
[root@vm1 ~]# ovs-vsctl set port br0 tag=100
[root@vm1 ~]# ovs-vsctl show
c152c245-2f6c-478c-9c07-2e4a3c7a2403
Bridge "br0"
Port "afc212d089e44_l"
tag: 200
Interface "afc212d089e44_l"
Port "1a18677188d94_l"
tag: 100
Interface "1a18677188d94_l"
Port "07bfe3100ddd4_l"
tag: 200
Interface "07bfe3100ddd4_l"
Port "br0"
tag: 100
Interface "br0"
type: internal
Port "6bf90751c96c4_l"
tag: 100
Interface "6bf90751c96c4_l"
ovs_version: "2.5.1"
测试, 此时con1和con2可以访问br0, con3和con4不能访问br0. 同样的道理,如果把tag设置为200, con3和con4就可以访问br0, 而con1和con2就不能访问br0.
[root@vm1 ~]# docker exec -it con1 ping -c 1 192.168.0.250
PING 192.168.0.250 (192.168.0.250): 56 data bytes
64 bytes from 192.168.0.250: seq=0 ttl=64 time=5.466 ms
--- 192.168.0.250 ping statistics ---
1 packets transmitted, 1 packets received, 0% packet loss
round-trip min/avg/max = 5.466/5.466/5.466 ms
[root@vm1 ~]# docker exec -it con2 ping -c 1 192.168.0.250
PING 192.168.0.250 (192.168.0.250): 56 data bytes
64 bytes from 192.168.0.250: seq=0 ttl=64 time=4.242 ms
--- 192.168.0.250 ping statistics ---
1 packets transmitted, 1 packets received, 0% packet loss
round-trip min/avg/max = 4.242/4.242/4.242 ms
[root@vm1 ~]# docker exec -it con3 ping -c 1 192.168.0.250
PING 192.168.0.250 (192.168.0.250): 56 data bytes
--- 192.168.0.250 ping statistics ---
1 packets transmitted, 0 packets received, 100% packet loss
[root@vm1 ~]# docker exec -it con4 ping -c 1 192.168.0.250
PING 192.168.0.250 (192.168.0.250): 56 data bytes
--- 192.168.0.250 ping statistics ---
1 packets transmitted, 0 packets received, 100% packet loss
4. 参考
1. Docker 容器与容器云
2. https://zpzhou.com/archives/openvswitch_vlan.html