基于docker 建立与主机同网段的zookeeper集群

一、需求简要说明

       1、通过在实体机上安装docker zookeeper来统一管理配置和服务；

       2、要求实体机与docker处于统一虚拟交换网，实现IP网段相同，用docker实现类似虚拟机的功能；

       3、该环境适用于小型应用系统，简化docker的网络维护；

二、逻辑结构

      1、测试环境：实体机mini centos7.2

      2、docker网络结构示意图 

       

三、配置过程脑图

四、主要填坑说明

       1、首次新建了docker容器后，用pipework分配了容器IP信息，宿主机可以ping通docker容器，进入容器也可以ping通宿主机和其它docker，但是重启后失效，重新pipework分配容器IP后，宿主机可以ping通docker容器，但是docker容器ping不通宿主机，通过ifconfig查看配置，显示正常，荣光ip route查看路由信息也正常；

       2、删除新建的docker容器并且把br0的配置删除，ens33网卡配置恢复之前的常规ethernet模式后，重新用pipework分配IP后，宿主机和docker容器又可以互通，在启动脚本中加了pipework的设置重启后仍然又是第一条现象；

       3、上述过程填坑：缺少了新建的docker到物理网关的bridge的配置，必须先建立新的bridge网络，然后绑定即可；

              新建bridge命令如下： 

docker network create --gateway 192.168.20.200 --subnet 192.168.20.0/24 -o com.docker.network.bridge.name=br0 --aux-address "DefaultGatewayIPv4=192.168.20.1" oak-net

主要参数说明：192.168.20.200----分配给br0的网络IP

om.docker.network.bridge.name=br0 ----绑定到br0

aux-address "DefaultGatewayIPv4=192.168.20.1" ----网关修改成外部的路由IP，而不用默认的宿主IP

oak-net----新建的bridge网络名称（新建后可以通过docker network list 命令查看到）

新建了birdge网络后即可用pipework分配和绑定docker 容器了

比如：

pipework br0 zookeeper-1 192.168.20.210/[email protected]

五、详细配置       

基于docker 建立与主机同网段的zookeeper集群

相关工具及步骤

docker

1、安装docker

yum install docker

2、启动docker并设定开机启动

docker start

systemctl enable docker

3、拉取zookeeper官方镜像

docker pull zookeeper(在可以上外网的主机上)

docker save -o /opt/zookeeper-image.tar zookeeper(在可以上外网的主机上)

docker load -i /opt/images/zookeeper-image.tar(在内网主机上)

4、用zookeeper镜像创建zookeeper容器集群

docker run --name zookeeper-1 --restart always -d zookeeper或者创建none模式容器:docker run --name zookeeper-1 --net=none --restart always -d zookeeper（此模式利用默认docker0网桥）

docker run --name zookeeper-2 --restart always -d zookeeper

docker run --name zookeeper-3 --restart always -d zookeeper

5、用zookeeper容器自带的客户端测试

docker exec -it zookeeper-1 zkCli.sh

pipework

1、下载并安装

yum install git -y

git clone https://github.com/jpetazzo/pipework

cd pipework

cp pipework /usr/bin/

2、修改linux 相关网卡配置信息

cd /etc/sysconfig/network-scripts/

cp ifcfg-ens33 ifcfg-br0

vi ifcfg-ens33

TYPE=Ethernet

PROXY_METHOD=none

BROWSER_ONLY=no

BOOTPROTO=static

DEFROUTE=yes

IPV4_FAILURE_FATAL=no

IPV6INIT=no

IPV6_AUTOCONF=no

IPV6_DEFROUTE=no

IPV6_FAILURE_FATAL=no

IPV6_ADDR_GEN_MODE=stable-privacy

NAME=ens33

UUID=fe661f96-ea5f-4a2a-aee5-849d7795dc3b

DEVICE=ens33

ONBOOT=yes

BRIDGE=br0

vi ifcfg-br0

TYPE=Bridge

BOOTPROTO=static

NAME=br0

DEVICE=br0

ONBOOT=yes

IPADDR=192.168.20.200

NETMASK=255.255.255.0

GATEWAY=192.168.20.1

service network restart

3、新建docker 到物理网关的bridge

docker network create --gateway 192.168.20.200 --subnet 192.168.20.0/24 -o com.docker.network.bridge.name=br0 --aux-address "DefaultGatewayIPv4=192.168.20.1" oak-net

4、新建和绑定上述容器IP

pipework br0 zookeeper-1 192.168.20.210/[email protected]

pipework br0 zookeeper-2 192.168.20.211/[email protected]

pipework br0 zookeeper-3 192.168.20.212/[email protected]

5、开机启动运行脚本设定

cd /etc/rc.d

vi rc.local

service docker start

pipework br0 zookeeper-1 192.168.20.220/[email protected]

chmod +x rc.local

brctl

1、安装虚拟网桥工具及查看信息

yum install bridge-utils

brctl show

相关工具及步骤

docker

1、安装docker

yum install docker

2、启动docker并设定开机启动

docker start

systemctl enable docker

3、拉取zookeeper官方镜像

docker pull zookeeper(在可以上外网的主机上)

docker save -o /opt/zookeeper-image.tar zookeeper(在可以上外网的主机上)

docker load -i /opt/images/zookeeper-image.tar(在内网主机上)

4、用zookeeper镜像创建zookeeper容器集群

docker run --name zookeeper-1 --restart always -d zookeeper或者创建none模式容器:docker run --name zookeeper-1 --net=none --restart always -d zookeeper（此模式利用默认docker0网桥）

docker run --name zookeeper-2 --restart always -d zookeeper

docker run --name zookeeper-3 --restart always -d zookeeper

5、用zookeeper容器自带的客户端测试

docker exec -it zookeeper-1 zkCli.sh

pipework

1、下载并安装

yum install git -y

git clone https://github.com/jpetazzo/pipework

cd pipework

cp pipework /usr/bin/

2、修改linux 相关网卡配置信息

cd /etc/sysconfig/network-scripts/

cp ifcfg-ens33 ifcfg-br0

vi ifcfg-ens33

TYPE=Ethernet

PROXY_METHOD=none

BROWSER_ONLY=no

BOOTPROTO=static

DEFROUTE=yes

IPV4_FAILURE_FATAL=no

IPV6INIT=no

IPV6_AUTOCONF=no

IPV6_DEFROUTE=no

IPV6_FAILURE_FATAL=no

IPV6_ADDR_GEN_MODE=stable-privacy

NAME=ens33

UUID=fe661f96-ea5f-4a2a-aee5-849d7795dc3b

DEVICE=ens33

ONBOOT=yes

BRIDGE=br0

vi ifcfg-br0

TYPE=Bridge

BOOTPROTO=static

NAME=br0

DEVICE=br0

ONBOOT=yes

IPADDR=192.168.20.200

NETMASK=255.255.255.0

GATEWAY=192.168.20.1

service network restart

3、新建docker 到物理网关的bridge

docker network create --gateway 192.168.20.200 --subnet 192.168.20.0/24 -o com.docker.network.bridge.name=br0 --aux-address "DefaultGatewayIPv4=192.168.20.1" oak-net

4、新建和绑定上述容器IP

pipework br0 zookeeper-1 192.168.20.210/[email protected]

pipework br0 zookeeper-2 192.168.20.211/[email protected]

pipework br0 zookeeper-3 192.168.20.212/[email protected]

5、开机启动运行脚本设定

cd /etc/rc.d

vi rc.local

service docker start

pipework br0 zookeeper-1 192.168.20.220/[email protected]

chmod +x rc.local

brctl

1、安装虚拟网桥工具及查看信息

yum install bridge-utils

brctl show

docker

1、安装docker

yum install docker

2、启动docker并设定开机启动

docker start

systemctl enable docker

3、拉取zookeeper官方镜像

docker pull zookeeper(在可以上外网的主机上)

docker save -o /opt/zookeeper-image.tar zookeeper(在可以上外网的主机上)

docker load -i /opt/images/zookeeper-image.tar(在内网主机上)

4、用zookeeper镜像创建zookeeper容器集群

docker run --name zookeeper-1 --restart always -d zookeeper或者创建none模式容器:docker run --name zookeeper-1 --net=none --restart always -d zookeeper（此模式利用默认docker0网桥）

docker run --name zookeeper-2 --restart always -d zookeeper

docker run --name zookeeper-3 --restart always -d zookeeper

5、用zookeeper容器自带的客户端测试

docker exec -it zookeeper-1 zkCli.sh

1、安装docker

yum install docker

2、启动docker并设定开机启动

docker start

systemctl enable docker

docker start

systemctl enable docker

3、拉取zookeeper官方镜像

docker pull zookeeper(在可以上外网的主机上)

docker save -o /opt/zookeeper-image.tar zookeeper(在可以上外网的主机上)

docker load -i /opt/images/zookeeper-image.tar(在内网主机上)

docker pull zookeeper(在可以上外网的主机上)

docker save -o /opt/zookeeper-image.tar zookeeper(在可以上外网的主机上)

docker load -i /opt/images/zookeeper-image.tar(在内网主机上)

4、用zookeeper镜像创建zookeeper容器集群

docker run --name zookeeper-1 --restart always -d zookeeper或者创建none模式容器:docker run --name zookeeper-1 --net=none --restart always -d zookeeper（此模式利用默认docker0网桥）

docker run --name zookeeper-2 --restart always -d zookeeper

docker run --name zookeeper-3 --restart always -d zookeeper

docker run --name zookeeper-1 --restart always -d zookeeper或者创建none模式容器:docker run --name zookeeper-1 --net=none --restart always -d zookeeper（此模式利用默认docker0网桥）

docker run --name zookeeper-2 --restart always -d zookeeper

docker run --name zookeeper-3 --restart always -d zookeeper

5、用zookeeper容器自带的客户端测试

docker exec -it zookeeper-1 zkCli.sh

docker exec -it zookeeper-1 zkCli.sh

pipework

1、下载并安装

yum install git -y

git clone https://github.com/jpetazzo/pipework

cd pipework

cp pipework /usr/bin/

2、修改linux 相关网卡配置信息

cd /etc/sysconfig/network-scripts/

cp ifcfg-ens33 ifcfg-br0

vi ifcfg-ens33

TYPE=Ethernet

PROXY_METHOD=none

BROWSER_ONLY=no

BOOTPROTO=static

DEFROUTE=yes

IPV4_FAILURE_FATAL=no

IPV6INIT=no

IPV6_AUTOCONF=no

IPV6_DEFROUTE=no

IPV6_FAILURE_FATAL=no

IPV6_ADDR_GEN_MODE=stable-privacy

NAME=ens33

UUID=fe661f96-ea5f-4a2a-aee5-849d7795dc3b

DEVICE=ens33

ONBOOT=yes

BRIDGE=br0

vi ifcfg-br0

TYPE=Bridge

BOOTPROTO=static

NAME=br0

DEVICE=br0

ONBOOT=yes

IPADDR=192.168.20.200

NETMASK=255.255.255.0

GATEWAY=192.168.20.1

service network restart

3、新建docker 到物理网关的bridge

docker network create --gateway 192.168.20.200 --subnet 192.168.20.0/24 -o com.docker.network.bridge.name=br0 --aux-address "DefaultGatewayIPv4=192.168.20.1" oak-net

4、新建和绑定上述容器IP

pipework br0 zookeeper-1 192.168.20.210/[email protected]

pipework br0 zookeeper-2 192.168.20.211/[email protected]

pipework br0 zookeeper-3 192.168.20.212/[email protected]

5、开机启动运行脚本设定

cd /etc/rc.d

vi rc.local

service docker start

pipework br0 zookeeper-1 192.168.20.220/[email protected]

chmod +x rc.local

1、下载并安装

yum install git -y

git clone https://github.com/jpetazzo/pipework

cd pipework

cp pipework /usr/bin/

yum install git -y

git clone https://github.com/jpetazzo/pipework

cd pipework

cp pipework /usr/bin/

2、修改linux 相关网卡配置信息

cd /etc/sysconfig/network-scripts/

cp ifcfg-ens33 ifcfg-br0

vi ifcfg-ens33

TYPE=Ethernet

PROXY_METHOD=none

BROWSER_ONLY=no

BOOTPROTO=static

DEFROUTE=yes

IPV4_FAILURE_FATAL=no

IPV6INIT=no

IPV6_AUTOCONF=no

IPV6_DEFROUTE=no

IPV6_FAILURE_FATAL=no

IPV6_ADDR_GEN_MODE=stable-privacy

NAME=ens33

UUID=fe661f96-ea5f-4a2a-aee5-849d7795dc3b

DEVICE=ens33

ONBOOT=yes

BRIDGE=br0

vi ifcfg-br0

TYPE=Bridge

BOOTPROTO=static

NAME=br0

DEVICE=br0

ONBOOT=yes

IPADDR=192.168.20.200

NETMASK=255.255.255.0

GATEWAY=192.168.20.1

service network restart

cd /etc/sysconfig/network-scripts/

cp ifcfg-ens33 ifcfg-br0

vi ifcfg-ens33

TYPE=Ethernet

PROXY_METHOD=none

BROWSER_ONLY=no

BOOTPROTO=static

DEFROUTE=yes

IPV4_FAILURE_FATAL=no

IPV6INIT=no

IPV6_AUTOCONF=no

IPV6_DEFROUTE=no

IPV6_FAILURE_FATAL=no

IPV6_ADDR_GEN_MODE=stable-privacy

NAME=ens33

UUID=fe661f96-ea5f-4a2a-aee5-849d7795dc3b

DEVICE=ens33

ONBOOT=yes

BRIDGE=br0

TYPE=Ethernet

PROXY_METHOD=none

BROWSER_ONLY=no

BOOTPROTO=static

DEFROUTE=yes

IPV4_FAILURE_FATAL=no

IPV6INIT=no

IPV6_AUTOCONF=no

IPV6_DEFROUTE=no

IPV6_FAILURE_FATAL=no

IPV6_ADDR_GEN_MODE=stable-privacy

NAME=ens33

UUID=fe661f96-ea5f-4a2a-aee5-849d7795dc3b

DEVICE=ens33

ONBOOT=yes

BRIDGE=br0

vi ifcfg-br0

TYPE=Bridge

BOOTPROTO=static

NAME=br0

DEVICE=br0

ONBOOT=yes

IPADDR=192.168.20.200

NETMASK=255.255.255.0

GATEWAY=192.168.20.1

TYPE=Bridge

BOOTPROTO=static

NAME=br0

DEVICE=br0

ONBOOT=yes

IPADDR=192.168.20.200

NETMASK=255.255.255.0

GATEWAY=192.168.20.1

service network restart

3、新建docker 到物理网关的bridge

docker network create --gateway 192.168.20.200 --subnet 192.168.20.0/24 -o com.docker.network.bridge.name=br0 --aux-address "DefaultGatewayIPv4=192.168.20.1" oak-net

docker network create --gateway 192.168.20.200 --subnet 192.168.20.0/24 -o com.docker.network.bridge.name=br0 --aux-address "DefaultGatewayIPv4=192.168.20.1" oak-net

4、新建和绑定上述容器IP

pipework br0 zookeeper-1 192.168.20.210/[email protected]

pipework br0 zookeeper-2 192.168.20.211/[email protected]

pipework br0 zookeeper-3 192.168.20.212/[email protected]

pipework br0 zookeeper-1 192.168.20.210/[email protected]

pipework br0 zookeeper-2 192.168.20.211/[email protected]

pipework br0 zookeeper-3 192.168.20.212/[email protected]

5、开机启动运行脚本设定

cd /etc/rc.d

vi rc.local

service docker start

pipework br0 zookeeper-1 192.168.20.220/[email protected]

chmod +x rc.local

cd /etc/rc.d

vi rc.local

service docker start

pipework br0 zookeeper-1 192.168.20.220/[email protected]

service docker start

pipework br0 zookeeper-1 192.168.20.220/[email protected]

chmod +x rc.local

brctl

1、安装虚拟网桥工具及查看信息

yum install bridge-utils

brctl show

1、安装虚拟网桥工具及查看信息

yum install bridge-utils

brctl show

yum install bridge-utils

brctl show

pipework br_name container_name ip/netmask@vlan(gateway)

yum install docker