[docker 网络] 单主机docker容器网络隔离 VLAN

0. 关于ovs VLAN

VLAN(Virtual Local Area Network)即虚拟局域网, 按照功能, 部门等因素将网络中的机器进行划分, 使之分属不同的部分, 每一个部分形成一个虚拟的局域网络, 共享一个单独的广域网. 将一个大型交换网络划分为许多个独立的广域网, 即VLAN.

目前OVS的端口支持四种VLAN模式,分别是trunk、access、native-tagged、native-untagged,刚创建的端口默认是不设置任何的VLAN模式。

模式 说明
默认 在默认模式下(VLAN_mode没被设置),如果指定了端口的tag属性,那么这个端口就工作在access模式,并且其trunk属性的值应该保持为空。否则,这个port就工作在trunk模式下,如果trunk被指定,则使用指定的trunk值。
trunk trunk模式的端口允许传输所有在其trunk属性中指定的那些VLAN对应的数据包。其他VLAN的数据包就会被丢弃。从trunk模式的端口中进入的数据包其VLAN ID不会发生变化。如果进入的数据包不含有VLAN ID,则该数据包进入交换机后的VLAN为0。从trunk模式的端口出去的数据包,如果VLAN ID不为空,则依然保持该VLAN ID,如果VLAN ID为空,则出去后不再包含802.1Q头部
access access模式的端口只允许不带VLAN的数据包进入,不管数据包的VLAN ID是否与其tag相同,只要含有VLAN ID,这个数据包都会被端口drop。数据包进入access端口后会被打上和端口tag相同的VLAN,而再从access端口出去时,数据包的VLAN会被删除,也就是说从access的端口出去的数据包和进来时一样是不带VLAN的。

来自于https://zpzhou.com/archives/openvswitch_vlan.html

[docker 网络] 单主机docker容器网络隔离 VLAN_第1张图片
来自容器与容器云.png

当PC1向PC3发送数据时, PC1将IP包封装在以太帧中, 帧的目的地址为PC3的地址, 此时帧中并没有tag信息. 当帧到达Port1时, Port1给帧打上tag(VID=100), 帧进入switch1, 然后帧通过Port3,Port4到达Switch2(假设Port3,Port4的trunks=[100,200], 所以Port3,Port4允许VLAN ID为100,200的帧通过). 在Switch2中, Port5所标记的VID和帧相同, MAC地址也匹配,帧就发送到Port5上, Port5将帧的tag信息去掉, 然后发送给PC3. 由于PC2,PC4与PC1的VLAN不同, 因此收不到PC1发出的帧.
此部分来自于容器与容器云

1. 前言

[docker 网络] 单主机docker容器网络隔离 VLAN_第2张图片
图片.png

本文将会利用ovs-docker,(关于ovs-docker可以参考[docker 网络] ovs-docker 使用及原理) 在同一台机器启动四个容器. 然后为容器加valn tag来进行容器隔离.

1.1 docker版本

[root@vm1 ~]# docker version
Client:
 Version:           18.09.6
 API version:       1.39
 Go version:        go1.10.8
 Git commit:        481bc77156
 Built:             Sat May  4 02:34:58 2019
 OS/Arch:           linux/amd64
 Experimental:      false

Server: Docker Engine - Community
 Engine:
  Version:          18.09.6
  API version:      1.39 (minimum version 1.12)
  Go version:       go1.10.8
  Git commit:       481bc77
  Built:            Sat May  4 02:02:43 2019
  OS/Arch:          linux/amd64
  Experimental:     false
[root@vm1 ~]# ovs-vsctl add-br br0
[root@vm1 ~]# ifconfig br0 192.168.0.250/24
[root@vm1 ~]# ovs-vsctl show
c152c245-2f6c-478c-9c07-2e4a3c7a2403
    Bridge "br0"
        Port "br0"
            Interface "br0"
                type: internal
    ovs_version: "2.5.1"

2. 配置

创建四个容器

[root@vm1 ~]# docker run -d --name con1 --net=none --privileged=true busybox top
30bdd285fc47e047e879a83f4a36a0c303ad6a38462da464b11b6266387684cd
[root@vm1 ~]# docker run -d --name con2 --net=none --privileged=true busybox top
eb0872370afd0ac1e544ae99522c4aff76abf0716283e4a41c6da471cb82ba21
[root@vm1 ~]# docker run -d --name con3 --net=none --privileged=true busybox top
db1a0b70e84e24d616d4f01c9518312794e12477779a4f8e2405427d754b0f94
[root@vm1 ~]# docker run -d --name con4 --net=none --privileged=true busybox top
247ef66e918147694f930db9e5f92afb8d2b4654b6f9959fb25ef5253bf9c887

利用ovs-docker给这些容器配置网络, 关于ovs-docker可以参考 [docker 网络] ovs-docker 使用及原理

[root@vm1 ~]# ovs-docker add-port br0 eth0 con1 --ipaddress=192.168.0.1/24 --gateway=192.168.0.250
[root@vm1 ~]# ovs-docker add-port br0 eth0 con2 --ipaddress=192.168.0.2/24 --gateway=192.168.0.250
[root@vm1 ~]# ovs-docker add-port br0 eth0 con3 --ipaddress=192.168.0.3/24 --gateway=192.168.0.250
[root@vm1 ~]# ovs-docker add-port br0 eth0 con4 --ipaddress=192.168.0.4/24 --gateway=192.168.0.250
[root@vm1 ~]# ovs-vsctl show
c152c245-2f6c-478c-9c07-2e4a3c7a2403
    Bridge "br0"
        Port "afc212d089e44_l"
            Interface "afc212d089e44_l"
        Port "1a18677188d94_l"
            Interface "1a18677188d94_l"
        Port "07bfe3100ddd4_l"
            Interface "07bfe3100ddd4_l"
        Port "br0"
            Interface "br0"
                type: internal
        Port "6bf90751c96c4_l"
            Interface "6bf90751c96c4_l"
    ovs_version: "2.5.1"
[root@vm1 ~]# 

2.1 测试

测试 : 从con1可以访问br0, con1, con2, con3, con4.

[root@vm1 ~]# docker exec -it con1 ping -c 1 192.168.0.250
PING 192.168.0.250 (192.168.0.250): 56 data bytes
64 bytes from 192.168.0.250: seq=0 ttl=64 time=5.501 ms

--- 192.168.0.250 ping statistics ---
1 packets transmitted, 1 packets received, 0% packet loss
round-trip min/avg/max = 5.501/5.501/5.501 ms
[root@vm1 ~]# docker exec -it con1 ping -c 1 192.168.0.1
PING 192.168.0.1 (192.168.0.1): 56 data bytes
64 bytes from 192.168.0.1: seq=0 ttl=64 time=0.053 ms

--- 192.168.0.1 ping statistics ---
1 packets transmitted, 1 packets received, 0% packet loss
round-trip min/avg/max = 0.053/0.053/0.053 ms
[root@vm1 ~]# docker exec -it con1 ping -c 1 192.168.0.2
PING 192.168.0.2 (192.168.0.2): 56 data bytes
64 bytes from 192.168.0.2: seq=0 ttl=64 time=4.844 ms

--- 192.168.0.2 ping statistics ---
1 packets transmitted, 1 packets received, 0% packet loss
round-trip min/avg/max = 4.844/4.844/4.844 ms
[root@vm1 ~]# docker exec -it con1 ping -c 1 192.168.0.3
PING 192.168.0.3 (192.168.0.3): 56 data bytes
64 bytes from 192.168.0.3: seq=0 ttl=64 time=5.708 ms

--- 192.168.0.3 ping statistics ---
1 packets transmitted, 1 packets received, 0% packet loss
round-trip min/avg/max = 5.708/5.708/5.708 ms
[root@vm1 ~]# docker exec -it con1 ping -c 1 192.168.0.4
PING 192.168.0.4 (192.168.0.4): 56 data bytes
64 bytes from 192.168.0.4: seq=0 ttl=64 time=4.738 ms

--- 192.168.0.4 ping statistics ---
1 packets transmitted, 1 packets received, 0% packet loss
round-trip min/avg/max = 4.738/4.738/4.738 ms
[root@vm1 ~]# 

别的容器也可以访问con1.

[root@vm1 ~]# docker exec -it con2 ping -c 1 192.168.0.1
PING 192.168.0.1 (192.168.0.1): 56 data bytes
64 bytes from 192.168.0.1: seq=0 ttl=64 time=4.524 ms

--- 192.168.0.1 ping statistics ---
1 packets transmitted, 1 packets received, 0% packet loss
round-trip min/avg/max = 4.524/4.524/4.524 ms
[root@vm1 ~]# docker exec -it con3 ping -c 1 192.168.0.1
PING 192.168.0.1 (192.168.0.1): 56 data bytes
64 bytes from 192.168.0.1: seq=0 ttl=64 time=5.441 ms

--- 192.168.0.1 ping statistics ---
1 packets transmitted, 1 packets received, 0% packet loss
round-trip min/avg/max = 5.441/5.441/5.441 ms
[root@vm1 ~]# docker exec -it con4 ping -c 1 192.168.0.1
PING 192.168.0.1 (192.168.0.1): 56 data bytes
64 bytes from 192.168.0.1: seq=0 ttl=64 time=4.805 ms

--- 192.168.0.1 ping statistics ---
1 packets transmitted, 1 packets received, 0% packet loss
round-trip min/avg/max = 4.805/4.805/4.805 ms
[root@vm1 ~]# 

3 加tag

为容器打上tag

[root@vm1 ~]# ovs-vsctl show
c152c245-2f6c-478c-9c07-2e4a3c7a2403
    Bridge "br0"
        Port "afc212d089e44_l"
            Interface "afc212d089e44_l"
        Port "1a18677188d94_l"
            Interface "1a18677188d94_l"
        Port "07bfe3100ddd4_l"
            Interface "07bfe3100ddd4_l"
        Port "br0"
            Interface "br0"
                type: internal
        Port "6bf90751c96c4_l"
            Interface "6bf90751c96c4_l"
    ovs_version: "2.5.1"

[root@vm1 ~]# ovs-vsctl list interface afc212d089e44_l | grep container_id
external_ids        : {container_id="con4", container_iface="eth0"}
[root@vm1 ~]# ovs-vsctl set port afc212d089e44_l tag=200

[root@vm1 ~]# ovs-vsctl list interface 1a18677188d94_l | grep container_id
external_ids        : {container_id="con2", container_iface="eth0"}
[root@vm1 ~]# ovs-vsctl set port 1a18677188d94_l tag=100

[root@vm1 ~]# ovs-vsctl list interface 07bfe3100ddd4_l | grep container_id
external_ids        : {container_id="con3", container_iface="eth0"}
[root@vm1 ~]# ovs-vsctl set port 07bfe3100ddd4_l tag=200

[root@vm1 ~]# ovs-vsctl list interface 6bf90751c96c4_l | grep container_id
external_ids        : {container_id="con1", container_iface="eth0"}
[root@vm1 ~]# ovs-vsctl set port 6bf90751c96c4_l tag=100

[root@vm1 ~]# ovs-vsctl show
c152c245-2f6c-478c-9c07-2e4a3c7a2403
    Bridge "br0"
        Port "afc212d089e44_l"
            tag: 200
            Interface "afc212d089e44_l"
        Port "1a18677188d94_l"
            tag: 100
            Interface "1a18677188d94_l"
        Port "07bfe3100ddd4_l"
            tag: 200
            Interface "07bfe3100ddd4_l"
        Port "br0"
            Interface "br0"
                type: internal
        Port "6bf90751c96c4_l"
            tag: 100
            Interface "6bf90751c96c4_l"
    ovs_version: "2.5.1"

3.1 测试

测试容器互通性
con1可以访问con2, 但是访问不了con3和con4.

[root@vm1 ~]# docker exec -it con1 ping -c 1 192.168.0.1
PING 192.168.0.1 (192.168.0.1): 56 data bytes
64 bytes from 192.168.0.1: seq=0 ttl=64 time=0.047 ms

--- 192.168.0.1 ping statistics ---
1 packets transmitted, 1 packets received, 0% packet loss
round-trip min/avg/max = 0.047/0.047/0.047 ms
[root@vm1 ~]# docker exec -it con1 ping -c 1 192.168.0.2
PING 192.168.0.2 (192.168.0.2): 56 data bytes
64 bytes from 192.168.0.2: seq=0 ttl=64 time=5.429 ms

--- 192.168.0.2 ping statistics ---
1 packets transmitted, 1 packets received, 0% packet loss
round-trip min/avg/max = 5.429/5.429/5.429 ms
[root@vm1 ~]# 
[root@vm1 ~]# docker exec -it con1 ping -c 1 192.168.0.3
PING 192.168.0.3 (192.168.0.3): 56 data bytes

--- 192.168.0.3 ping statistics ---
1 packets transmitted, 0 packets received, 100% packet loss
[root@vm1 ~]# docker exec -it con1 ping -c 1 192.168.0.4
PING 192.168.0.4 (192.168.0.4): 56 data bytes

--- 192.168.0.4 ping statistics ---
1 packets transmitted, 0 packets received, 100% packet loss

con2 可以访问con1, 但是con2 不能访问con3和con4

[root@vm1 ~]# docker exec -it con2 ping -c 1 192.168.0.2
PING 192.168.0.2 (192.168.0.2): 56 data bytes
64 bytes from 192.168.0.2: seq=0 ttl=64 time=0.062 ms

--- 192.168.0.2 ping statistics ---
1 packets transmitted, 1 packets received, 0% packet loss
round-trip min/avg/max = 0.062/0.062/0.062 ms
[root@vm1 ~]# docker exec -it con2 ping -c 1 192.168.0.1
PING 192.168.0.1 (192.168.0.1): 56 data bytes
64 bytes from 192.168.0.1: seq=0 ttl=64 time=3.548 ms

--- 192.168.0.1 ping statistics ---
1 packets transmitted, 1 packets received, 0% packet loss
round-trip min/avg/max = 3.548/3.548/3.548 ms
[root@vm1 ~]# docker exec -it con2 ping -c 1 192.168.0.3
PING 192.168.0.3 (192.168.0.3): 56 data bytes

--- 192.168.0.3 ping statistics ---
1 packets transmitted, 0 packets received, 100% packet loss
[root@vm1 ~]# docker exec -it con2 ping -c 1 192.168.0.4
PING 192.168.0.4 (192.168.0.4): 56 data bytes

--- 192.168.0.4 ping statistics ---
1 packets transmitted, 0 packets received, 100% packet loss
[root@vm1 ~]# 

con3不能访问con1和con2, 但是可以访问con4

[root@vm1 ~]# docker exec -it con3 ping -c 1 192.168.0.1
PING 192.168.0.1 (192.168.0.1): 56 data bytes

--- 192.168.0.1 ping statistics ---
1 packets transmitted, 0 packets received, 100% packet loss
[root@vm1 ~]# docker exec -it con3 ping -c 1 192.168.0.2
PING 192.168.0.2 (192.168.0.2): 56 data bytes

--- 192.168.0.2 ping statistics ---
1 packets transmitted, 0 packets received, 100% packet loss
[root@vm1 ~]# docker exec -it con3 ping -c 1 192.168.0.3
PING 192.168.0.3 (192.168.0.3): 56 data bytes
64 bytes from 192.168.0.3: seq=0 ttl=64 time=0.054 ms

--- 192.168.0.3 ping statistics ---
1 packets transmitted, 1 packets received, 0% packet loss
round-trip min/avg/max = 0.054/0.054/0.054 ms
[root@vm1 ~]# docker exec -it con3 ping -c 1 192.168.0.4
PING 192.168.0.4 (192.168.0.4): 56 data bytes
64 bytes from 192.168.0.4: seq=0 ttl=64 time=3.399 ms

--- 192.168.0.4 ping statistics ---
1 packets transmitted, 1 packets received, 0% packet loss
round-trip min/avg/max = 3.399/3.399/3.399 ms
[root@vm1 ~]# 

con4 不能访问 con1 和 con2, 但是可以访问con3.

[root@vm1 ~]# docker exec -it con4 ping -c 1 192.168.0.1
PING 192.168.0.1 (192.168.0.1): 56 data bytes

--- 192.168.0.1 ping statistics ---
1 packets transmitted, 0 packets received, 100% packet loss
[root@vm1 ~]# docker exec -it con4 ping -c 1 192.168.0.2
PING 192.168.0.2 (192.168.0.2): 56 data bytes

--- 192.168.0.2 ping statistics ---
1 packets transmitted, 0 packets received, 100% packet loss
[root@vm1 ~]# docker exec -it con4 ping -c 1 192.168.0.3
PING 192.168.0.3 (192.168.0.3): 56 data bytes
64 bytes from 192.168.0.3: seq=0 ttl=64 time=5.234 ms

--- 192.168.0.3 ping statistics ---
1 packets transmitted, 1 packets received, 0% packet loss
round-trip min/avg/max = 5.234/5.234/5.234 ms
[root@vm1 ~]# docker exec -it con4 ping -c 1 192.168.0.4
PING 192.168.0.4 (192.168.0.4): 56 data bytes
64 bytes from 192.168.0.4: seq=0 ttl=64 time=0.065 ms

--- 192.168.0.4 ping statistics ---
1 packets transmitted, 1 packets received, 0% packet loss
round-trip min/avg/max = 0.065/0.065/0.065 ms
[root@vm1 ~]# 

3.2 访问br0

访问br0, con1,con2,con3,con4均不能访问br0

[root@vm1 ~]# docker exec -it con1 ping -c 1 192.168.0.250
PING 192.168.0.250 (192.168.0.250): 56 data bytes

--- 192.168.0.250 ping statistics ---
1 packets transmitted, 0 packets received, 100% packet loss
[root@vm1 ~]# 
[root@vm1 ~]# docker exec -it con2 ping -c 1 192.168.0.250
PING 192.168.0.250 (192.168.0.250): 56 data bytes

--- 192.168.0.250 ping statistics ---
1 packets transmitted, 0 packets received, 100% packet loss
[root@vm1 ~]# docker exec -it con3 ping -c 1 192.168.0.250
PING 192.168.0.250 (192.168.0.250): 56 data bytes

--- 192.168.0.250 ping statistics ---
1 packets transmitted, 0 packets received, 100% packet loss
[root@vm1 ~]# docker exec -it con4 ping -c 1 192.168.0.250
PING 192.168.0.250 (192.168.0.250): 56 data bytes

--- 192.168.0.250 ping statistics ---
1 packets transmitted, 0 packets received, 100% packet loss

为br0设置tag

[root@vm1 ~]# ovs-vsctl set port br0 tag=100
[root@vm1 ~]# ovs-vsctl show
c152c245-2f6c-478c-9c07-2e4a3c7a2403
    Bridge "br0"
        Port "afc212d089e44_l"
            tag: 200
            Interface "afc212d089e44_l"
        Port "1a18677188d94_l"
            tag: 100
            Interface "1a18677188d94_l"
        Port "07bfe3100ddd4_l"
            tag: 200
            Interface "07bfe3100ddd4_l"
        Port "br0"
            tag: 100
            Interface "br0"
                type: internal
        Port "6bf90751c96c4_l"
            tag: 100
            Interface "6bf90751c96c4_l"
    ovs_version: "2.5.1"

测试, 此时con1和con2可以访问br0, con3和con4不能访问br0. 同样的道理,如果把tag设置为200, con3和con4就可以访问br0, 而con1和con2就不能访问br0.

[root@vm1 ~]# docker exec -it con1 ping -c 1 192.168.0.250
PING 192.168.0.250 (192.168.0.250): 56 data bytes
64 bytes from 192.168.0.250: seq=0 ttl=64 time=5.466 ms

--- 192.168.0.250 ping statistics ---
1 packets transmitted, 1 packets received, 0% packet loss
round-trip min/avg/max = 5.466/5.466/5.466 ms
[root@vm1 ~]# docker exec -it con2 ping -c 1 192.168.0.250
PING 192.168.0.250 (192.168.0.250): 56 data bytes
64 bytes from 192.168.0.250: seq=0 ttl=64 time=4.242 ms

--- 192.168.0.250 ping statistics ---
1 packets transmitted, 1 packets received, 0% packet loss
round-trip min/avg/max = 4.242/4.242/4.242 ms
[root@vm1 ~]# docker exec -it con3 ping -c 1 192.168.0.250
PING 192.168.0.250 (192.168.0.250): 56 data bytes

--- 192.168.0.250 ping statistics ---
1 packets transmitted, 0 packets received, 100% packet loss
[root@vm1 ~]# docker exec -it con4 ping -c 1 192.168.0.250
PING 192.168.0.250 (192.168.0.250): 56 data bytes

--- 192.168.0.250 ping statistics ---
1 packets transmitted, 0 packets received, 100% packet loss

4. 参考

1. Docker 容器与容器云
2. https://zpzhou.com/archives/openvswitch_vlan.html

你可能感兴趣的:([docker 网络] 单主机docker容器网络隔离 VLAN)