Open-vSwitch构建Docker跨主机网络
环境说明：
| 主机名 | 操作系统 | 宿主机IP | Docker IP |
|---|---|---|---|
| ovs01 | ubuntu 18.04 | 192.168.168.10 | 172.17.0.1 |
| ovs02 | ubuntu 18.04 | 192.168.168.11 | 172.17.1.2 |
安装配置OVS网络：
1、安装docker－ce (安装过程此处略)
2、设置docker0网段(ovs02同样操作，bip不同)
$ sudo vi /etc/docker/daemon.json
{
"bip":"172.17.0.1/24"
}
$ sudo systemctl restart docker3、安装openvswitch-switch和bridge-utils
$ sudo apt-get -y install openvswitch-switch bridge-utils4、查看ovs运行状态
$ sudo ps -ea | grep ovs
1526 ? 00:00:00 ovsdb-server
1593 ? 00:00:00 ovs-vswitchd5、查看ovs版本信息和ovs支持的OpenFlow协议的版本
$ sudo ovs-appctl --version
ovs-appctl (Open vSwitch) 2.9.5
$ sudo ovs-ofctl --version
ovs-ofctl (Open vSwitch) 2.9.5
OpenFlow versions 0x1:0x56、创建br0网桥并激活
$ sudo ovs-vsctl add-br br0
$ sudo ip link set dev br0 up7、创建gre隧道(remote_ip为peer宿主机ip)
$ sudo ovs-vsctl add-port br0 gre0 -- set Interface gre0 type=gre options:remote_ip=192.168.168.11 //ovs01配置
$ sudo ovs-vsctl add-port br0 gre0 -- set Interface gre0 type=gre options:remote_ip=192.168.168.10 //ovs02配置注：如有多台docker主机需要构建网络创建多个gre隧道
8、将br0作为接口加入docker0网桥
$ sudo brctl addif docker0 br0
$ sudo brctl stp docker0 on9、查看网桥配置
$ sudo ovs-vsctl show
cedc63c1-97d6-4e5e-bdf0-3efc0a5b7aa4
Bridge "br0"
Port "br0"
Interface "br0"
type: internal
Port "vxlan0"
Interface "gre0"
type: gre
options: {remote_ip="192.168.168.11"}
ovs_version: "2.9.5"
$ brctl show
bridge name bridge id STP enabled interfaces
docker0 8000.02425f251c20 no br010、添加静态路由(网段地址为peer Docker网段)
$ sudo ip route add 172.17.1.0/24 dev docker0 //ovs01添加peer docker net
$ sudo ip route add 172.17.0.0/24 dev docker0 //ovs02添加peer docker net11、测试连通性
$ docker run -it busybox:1.28.3 /bin/sh //ovs01测试
/ # ip a
1: lo: mtu 65536 qdisc noqueue qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
2: gre0@NONE: mtu 1476 qdisc noop qlen 1000
link/gre 0.0.0.0 brd 0.0.0.0
3: gretap0@NONE: mtu 1462 qdisc noop qlen 1000
link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff
4: erspan0@NONE: mtu 1450 qdisc noop qlen 1000
link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff
10: eth0@if11: mtu 1500 qdisc noqueue
link/ether 02:42:ac:11:00:02 brd ff:ff:ff:ff:ff:ff
inet 172.17.0.2/24 brd 172.17.0.255 scope global eth0
valid_lft forever preferred_lft forever
/ # ping 172.17.1.2
PING 172.17.1.2 (172.17.1.2): 56 data bytes
64 bytes from 172.17.1.2: seq=0 ttl=63 time=3.302 ms
64 bytes from 172.17.1.2: seq=1 ttl=63 time=0.824 ms
$ docker run -it busybox:1.28.3 /bin/sh //ovs02测试
/ # ip a
1: lo: mtu 65536 qdisc noqueue qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
2: gre0@NONE: mtu 1476 qdisc noop qlen 1000
link/gre 0.0.0.0 brd 0.0.0.0
3: gretap0@NONE: mtu 1462 qdisc noop qlen 1000
link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff
4: erspan0@NONE: mtu 1450 qdisc noop qlen 1000
link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff
10: eth0@if11: mtu 1500 qdisc noqueue
link/ether 02:42:ac:11:01:02 brd ff:ff:ff:ff:ff:ff
inet 172.17.1.2/24 brd 172.17.1.255 scope global eth0
valid_lft forever preferred_lft forever
/ # ping 172.17.0.2
PING 172.17.0.2 (172.17.0.2): 56 data bytes
64 bytes from 172.17.0.2: seq=0 ttl=63 time=1.903 ms
64 bytes from 172.17.0.2: seq=1 ttl=63 time=0.765 ms 12、网桥配置和添加路由配置重启宿主机后会失效，写成shell脚本，重启后执行
$ sudo cat > add_bridge.sh < add_bridge.sh < CentOS7 OVS安装并生成RPM安装包
1、安装依赖包
yum -y install make gcc openssl-devel autoconf automake rpm-build redhat-rpm-config python-devel kernel-devel kernel-debug-devel libtool bridge-utils2、下载OVS二进制安装包
# mkdir -p ~/rpmbuild/SOURCES
# wget https://www.openvswitch.org/releases/openvswitch-2.5.9.tar.gz -P ~/rpmbuild/SOURCES3、生成OVS RPM安装包
# cd ~/rpmbuild/SOURCES
# tar -xvf openvswitch-2.5.9.tar.gz
# sed 's/openvswitch-kmod, //g' openvswitch-2.5.9/rhel/openvswitch.spec > openvswitch-2.5.9/rhel/openvswitch_no_kmod.spec
# rpmbuild -bb --nocheck openvswitch-2.12.0/rhel/openvswitch_no_kmod.spec4、安装OVS
# yum localinstall ~/rpmbuild/RPMS/x86_64/openvswitch-2.5.9-1.x86_64.rpm下载备份OVS RPM包，可在其它CentOS系统直接使用
5、启动OVS服务
# service openvswitch start
Starting openvswitch (via systemctl): [ OK ]
# service openvswitch status
ovsdb-server is running with pid 7004
ovs-vswitchd is running with pid 7024
# chkconfig --add openvswitch
# chkconfig openvswitch on
# tail -50f /var/log/messages
Jan 19 11:07:39 ovs yum[6922]: Installed: openvswitch-2.5.9-1.x86_64
Jan 19 11:07:52 ovs systemd: Starting LSB: Open vSwitch switch...
Jan 19 11:07:52 ovs openvswitch: /etc/openvswitch/conf.db does not exist ... (warning).
Jan 19 11:07:52 ovs openvswitch: Creating empty database /etc/openvswitch/conf.db [ OK ]
Jan 19 11:07:52 ovs openvswitch: Starting ovsdb-server [ OK ]
Jan 19 11:07:52 ovs ovs-vsctl: ovs|00001|vsctl|INFO|Called as ovs-vsctl --no-wait -- init -- set Open_vSwitch . db-version=7.12.1
Jan 19 11:07:52 ovs ovs-vsctl: ovs|00001|vsctl|INFO|Called as ovs-vsctl --no-wait set Open_vSwitch . ovs-version=2.5.9 "external-ids:system-id=\"5aed6a14-bad2-438b-b012-c3dcbcb817fc\"" "system-type=\"unknown\"" "system-version=\"unknown\""
Jan 19 11:07:52 ovs openvswitch: Configuring Open vSwitch system IDs [ OK ]
Jan 19 11:07:52 ovs kernel: nf_conntrack version 0.5.0 (7928 buckets, 31712 max)
Jan 19 11:07:52 ovs kernel: openvswitch: Open vSwitch switching datapath
Jan 19 11:07:52 ovs openvswitch: Inserting openvswitch module [ OK ]
Jan 19 11:07:52 ovs openvswitch: Starting ovs-vswitchd [ OK ]
Jan 19 11:07:52 ovs openvswitch: Enabling remote OVSDB managers [ OK ]
Jan 19 11:07:52 ovs systemd: Started LSB: Open vSwitch switch.