Centos 7.2 系统下安装部署Docker工具 Swarm环境

1、环境

    IP                   服务        主机名
    192.168.3.204   Swarm管理节点   controler
    192.168.3.202   Swarm的node节点    node2
    192.168.3.201   Swarm的node节点    node1

2、配置docker yum源（三台机器）

    cd /etc/yum.repos.d/
    touch docker.repo
    vim docker.repo

添加内容如下：*

    [dockerrepo]
    name=Docker Repository
    baseurl=https://yum.dockerproject.org/repo/main/centos/$releasever/
    enabled=1
    gpgcheck=1
    gpgkey=https://yum.dockerproject.org/gpg

:wq保存退出

3、配置hosts表（三台机器）

    vim /etc/hosts

添加内容如下：

    192.168.3.204   controller
    192.168.3.201    node1
    192.168.3.202    node2

4、配置manager节点（192.168.3.204）

4.1、删除旧版本的环境

    yum remove docker docker-client docker-client-latest docker-common docker-latest docker-latest-logrotate docker-logrotate docker-selinux docker-engine-selinux docker-engine

4.2、查看可用的docker版本

    [root@controller lib]#  yum list docker-engine.x86_64  --showduplicates |sort -r
    Loading mirror speeds from cached hostfile
    Loaded plugins: fastestmirror, langpacks
    docker-engine.x86_64             1.9.1-1.el7.centos                   dockerrepo
    docker-engine.x86_64             1.9.0-1.el7.centos                   dockerrepo
    docker-engine.x86_64             1.8.3-1.el7.centos                   dockerrepo
    docker-engine.x86_64             1.8.2-1.el7.centos                   dockerrepo
    docker-engine.x86_64             1.8.1-1.el7.centos                   dockerrepo
    docker-engine.x86_64             1.8.0-1.el7.centos                   dockerrepo
    docker-engine.x86_64             1.7.1-1.el7.centos                   dockerrepo
    docker-engine.x86_64             17.05.0.ce-1.el7.centos              dockerrepo
    docker-engine.x86_64             17.04.0.ce-1.el7.centos              dockerrepo
    docker-engine.x86_64             17.03.1.ce-1.el7.centos              dockerrepo
    docker-engine.x86_64             17.03.0.ce-1.el7.centos              dockerrepo
    docker-engine.x86_64             1.7.0-1.el7.centos                   dockerrepo
    docker-engine.x86_64             1.13.1-1.el7.centos                  dockerrepo
    docker-engine.x86_64             1.13.0-1.el7.centos                  dockerrepo
    docker-engine.x86_64             1.12.6-1.el7.centos                  dockerrepo
    docker-engine.x86_64             1.12.5-1.el7.centos                  dockerrepo
    docker-engine.x86_64             1.12.4-1.el7.centos                  dockerrepo
    docker-engine.x86_64             1.12.3-1.el7.centos                  dockerrepo
    docker-engine.x86_64             1.12.2-1.el7.centos                  dockerrepo
    docker-engine.x86_64             1.12.1-1.el7.centos                  dockerrepo
    docker-engine.x86_64             1.12.0-1.el7.centos                  dockerrepo
    docker-engine.x86_64             1.11.2-1.el7.centos                  dockerrepo
    docker-engine.x86_64             1.11.1-1.el7.centos                  dockerrepo
    docker-engine.x86_64             1.11.0-1.el7.centos                  dockerrepo
    docker-engine.x86_64             1.10.3-1.el7.centos                  dockerrepo
    docker-engine.x86_64             1.10.2-1.el7.centos                  dockerrepo
    docker-engine.x86_64             1.10.1-1.el7.centos                  dockerrepo
    docker-engine.x86_64             1.10.0-1.el7.centos                  dockerrepo
    Available Packages

4.3、安装稳定的版本1.13.1

    [root@controller lib]# yum -y install docker-engine-1.13.1-1.el7.centos 

启动docker服务并设置开机自启动

    systemctl start docker
    systemctl enable docker

4.4、下载swarm镜像

    [root@controller ~]# docker pull swarm
    Using default tag: latest
    latest: Pulling from library/swarm
    Digest: sha256:406022f04a3d0c5ce4dbdb60422f24052c20ab7e6d41ebe5723aa649c3833975
    Status: Downloaded newer image for swarm:latest
    [root@controller ~]# docker images
    REPOSITORY          TAG                 IMAGE ID            CREATED             SIZE
    swarm               latest              ff454b4a0e84        7 weeks ago         12.7 MB

4.5、创建swarm

    [root@controller ~]# docker swarm init --advertise-addr 192.168.3.204
    Swarm initialized: current node (r344b4bz1ug3bnw6olsxzqtis) is now a manager.


    To add a worker to this swarm, run the following command:


        docker swarm join \
        --token SWMTKN-1-1kvrsn7j74ie7fqxzxb0wndk76gjnwdyc6jnhe4771h55edgzw-20spfbubw7ckh1w0w3plvsak3 \
        192.168.3.204:2377


    To add a manager to this swarm, run 'docker swarm join-token manager' and follow the instructions.

注：上面的token值要保存，其他节点均通过它加入集群

4.6、查看集群信息

    [root@controller ~]# docker info
    Containers: 0
     Running: 0
     Paused: 0
     Stopped: 0
    Images: 1
    Server Version: 1.13.1
    Storage Driver: overlay
     Backing Filesystem: xfs
     Supports d_type: false
    Logging Driver: json-file
    Cgroup Driver: cgroupfs
    Plugins: 
     Volume: local
     Network: bridge host macvlan null overlay
    Swarm: active
     NodeID: r344b4bz1ug3bnw6olsxzqtis
     Is Manager: true
     ClusterID: p5r0a72jdoq9cztshyf0bvfye
     Managers: 1
     Nodes: 1
     Orchestration:
      Task History Retention Limit: 5
     Raft:
      Snapshot Interval: 10000
      Number of Old Snapshots to Retain: 0
      Heartbeat Tick: 1
      Election Tick: 3
     Dispatcher:
      Heartbeat Period: 5 seconds
     CA Configuration:
      Expiry Duration: 3 months
     Node Address: 192.168.3.204
     Manager Addresses:
      192.168.3.204:2377
    Runtimes: runc
    Default Runtime: runc
    Init Binary: docker-init
    containerd version: aa8187dbd3b7ad67d8e5e3a15115d3eef43a7ed1
    runc version: 9df8b306d01f59d3a8029be411de015b7304dd8f
    init version: 949e6fa
    Security Options:
     seccomp
      Profile: default
    Kernel Version: 3.10.0-327.18.2.el7.x86_64
    Operating System: CentOS Linux 7 (Core)
    OSType: linux
    Architecture: x86_64
    CPUs: 4
    Total Memory: 7.551 GiB
    Name: controller
    ID: 5E6V:BHYM:3L74:EYS3:ZUID:INQP:DINI:U7I6:QKJZ:UCYY:QZJF:GXGN
    Docker Root Dir: /var/lib/docker
    Debug Mode (client): false
    Debug Mode (server): false
    Registry: https://index.docker.io/v1/
    Experimental: false
    Insecure Registries:
     127.0.0.0/8
    Live Restore Enabled: false

4.7、查看集群节点信息

    [root@controller ~]# docker node ls
    ID                           HOSTNAME    STATUS  AVAILABILITY  MANAGER STATUS
    r344b4bz1ug3bnw6olsxzqtis *  controller  Ready   Active        Leader

5、配置节点（192.168.3.202,192.168.3.201）

5.1、安装docker

    yum -y install docker-engine-1.13.1-1.el7.centos

启动docker

    systemctl start docker

开机自启动

    systemctl enable docker

5.2、加入到集群

    docker swarm join --token SWMTKN-1-1kvrsn7j74ie7fqxzxb0wndk76gjnwdyc6jnhe4771h55edgzw-20spfbubw7ckh1w0w3plvsak3 192.168.3.204:2377

5.3、查看docker信息（两个节点一样执行，我这里只贴出在node2执行的信息）

    [root@node2 ~]# docker info
    Containers: 0
     Running: 0
     Paused: 0
     Stopped: 0
    Images: 0
    Server Version: 1.13.1
    Storage Driver: overlay
     Backing Filesystem: xfs
     Supports d_type: false
    Logging Driver: json-file
    Cgroup Driver: cgroupfs
    Plugins: 
     Volume: local
     Network: bridge host macvlan null overlay
    Swarm: active
     NodeID: pdqau3dpxtgnr6gg78zbt7nnz
     Is Manager: false
     Node Address: 192.168.3.202
     Manager Addresses:
      192.168.3.204:2377
    Runtimes: runc
    Default Runtime: runc
    Init Binary: docker-init
    containerd version: aa8187dbd3b7ad67d8e5e3a15115d3eef43a7ed1
    runc version: 9df8b306d01f59d3a8029be411de015b7304dd8f
    init version: 949e6fa
    Security Options:
     seccomp
      Profile: default
    Kernel Version: 3.10.0-229.el7.x86_64
    Operating System: CentOS Linux 7 (Core)
    OSType: linux
    Architecture: x86_64
    CPUs: 2
    Total Memory: 1.797 GiB
    Name: node2
    ID: Q7VU:TMEP:NCO6:YVXC:O6U2:2DON:5RUL:VGJ4:D2TA:345M:I4Q6:UHN5
    Docker Root Dir: /var/lib/docker
    Debug Mode (client): false
    Debug Mode (server): false
    Registry: https://index.docker.io/v1/
    WARNING: bridge-nf-call-ip6tables is disabled
    Experimental: false
    Insecure Registries:
     127.0.0.0/8
    Live Restore Enabled: false

5.4、在集群节点（192.168.3.204）查看节点信息

    [root@controller ~]# docker node ls
    ID                           HOSTNAME    STATUS  AVAILABILITY  MANAGER STATUS
    hjba4ers0ahzbdvggajc3b73s    node1       Ready   Active        
    pdqau3dpxtgnr6gg78zbt7nnz    node2       Ready   Active        
    r344b4bz1ug3bnw6olsxzqtis *  controller  Ready   Active        Leader

6、在swarm中部署nginx服务

6.1、创建覆盖网络（保障不同主机容器互通）

    [root@controller ~]# docker network create -d overlay nginx
    692ud2yyqimkuq6rufpnwnt05
    [root@controller ~]# docker network ls
    NETWORK ID          NAME                DRIVER              SCOPE
    27940e4bc00a        bridge              bridge              local
    cd675706a24e        docker_gwbridge     bridge              local
    ec66ee0e74b1        host                host                local
    l8d4j4w5o52u        ingress             overlay             swarm
    692ud2yyqimk        nginx               overlay             swarm
    3a44f24d7c1f        none                null                local

6.2、创建服务

创建：

    [root@controller ~]# docker service create --replicas 1 --network nginx --name MyNginx -p 80:80 nginx
    8sk44dipvooy4sb2e7e4l8wb1

查看服务

    [root@controller ~]# docker service ls
    ID            NAME     MODE        REPLICAS  IMAGE
    8sk44dipvooy  MyNginx  replicated  1/1       nginx:latest

查看摘要信息：

    [root@controller ~]#  docker service inspect --pretty MyNginx


    ID:        8sk44dipvooy4sb2e7e4l8wb1
    Name:        MyNginx
    Service Mode:    Replicated
     Replicas:    1
    Placement:
    UpdateConfig:
     Parallelism:    1
     On failure:    pause
     Max failure ratio: 0
    ContainerSpec:
     Image:        nginx:latest@sha256:d85914d547a6c92faa39ce7058bd7529baacab7e0cd4255442b04577c4d1f424
    Resources:
    Networks: nginx 
    Endpoint Mode:    vip
    Ports:
     PublishedPort 80
      Protocol = tcp
      TargetPort = 80 

6.3、查看服务的运行情况

    [root@controller ~]# docker service ps MyNginx
    ID            NAME       IMAGE         NODE   DESIRED STATE  CURRENT STATE               ERROR  PORTS
    khvp4r03de4f  MyNginx.1  nginx:latest  node2  Running        Running about a minute ago       

6.4、访问服务

浏览器或者curl访问服务http://192.168.3.204

    
    
    
    
    
    
    
    
Welcome to nginx!
    
If you see this page, the nginx web server is successfully installed and
    working. Further configuration is required.


    
For online documentation and support please refer to
    nginx.org.

    Commercial support is available at
    nginx.com.


    
Thank you for using nginx.
    
    

7、Service的复制功能

7.1、增加

设置5个副本

    [root@controller ~]# docker service scale MyNginx=5
    MyNginx scaled to 5
    [root@controller ~]# docker service ps MyNginx
    ID            NAME       IMAGE         NODE        DESIRED STATE  CURRENT STATE            ERROR  PORTS
    khvp4r03de4f  MyNginx.1  nginx:latest  node2       Running        Running 12 minutes ago          
    wecu7jye2mg3  MyNginx.2  nginx:latest  controller  Running        Preparing 5 seconds ago         
    5a6rp8c6arvt  MyNginx.3  nginx:latest  node2       Running        Running 4 seconds ago           
    68lu0fpg75on  MyNginx.4  nginx:latest  node1       Running        Preparing 5 seconds ago         
    todza2i5lnnt  MyNginx.5  nginx:latest  controller  Running        Preparing 5 seconds ago       

7.2、减少

变更为1个副本

    [root@controller ~]# docker service scale MyNginx=1
    MyNginx scaled to 1
    [root@controller ~]# docker service ls
    ID            NAME     MODE        REPLICAS  IMAGE
    8sk44dipvooy  MyNginx  replicated  1/1       nginx:latest
    [root@controller ~]# docker service ps MyNginx
    ID            NAME           IMAGE         NODE        DESIRED STATE  CURRENT STATE           ERROR                             PORTS
    exoh860irad5  MyNginx.1      nginx:latest  node1       Running        Running 4 minutes ago         

8、使用volume

8.1、查看volume

    [root@controller ~]# docker volume ls
    DRIVER              VOLUME NAME

8.2、创建volume

    [root@controller ~]# docker volume create --name nginx
    nginx
    [root@controller ~]# docker volume ls
    DRIVER              VOLUME NAME
    local               nginx

8.3、创建服务并指定挂载

src是volume名称

dst是挂载点（也就是容器内的路径，如果不存在会自动创建）

注：若指定的是本机的路径，需要把type改成bind, src是本机的路径

    [root@controller ~]# docker service create  --replicas 2 --mount type=volume,src=nginx,dst=/farmer --name ng nginx
    r7bld5xg9cy6285z6207wlly2
    [root@controller ~]# docker service ls
    ID            NAME     MODE        REPLICAS  IMAGE
    9et4o10fn0w2  MyNginx  replicated  2/2       nginx:latest
    r7bld5xg9cy6  ng       replicated  2/2       nginx:latest
    [root@controller ~]# docker service ps ng
    ID            NAME  IMAGE         NODE   DESIRED STATE  CURRENT STATE           ERROR  PORTS
    190gm96z6brc  ng.1  nginx:latest  node2  Running        Running 25 seconds ago         
    093pql11ttdm  ng.2  nginx:latest  node1  Running        Running 25 seconds ago         

9、创建nginx服务和负载均衡的测试

9.1、创建volume名称为modules

    docker volume create --name modules

注：因为/etc/nginx/mudules是软连接到/usr/lib/nginx/modules，一旦将/etc/nginx挂载到本地，这个目录将会失效，所以需要在modules也挂到本地

    lrwxrwxrwx 1 root root   22 Jul 24 21:02 modules -> /usr/lib/nginx/modules 

9.2、创建软连接（所有节点，192.168.3.204,192.168.3.202,192.168.3.201）

    mkdir /usr/lib/nginx
    ln -s /var/lib/docker/volumes/modules/_data/modules/ /usr/lib/nginx/modules

9.3、创建test服务

    [root@controller ~]# docker service create --replicas 2 --mount type=volume,src=nginx,dst=/etc/nginx/ --mount type=volume,dst=/usr/lib/nginx/,src=modules --name test -p 8880:80 nginx
    9mt9wuabsluykwfh2j7ez9gut
    [root@controller ~]# docker service ps test
    ID            NAME    IMAGE         NODE        DESIRED STATE  CURRENT STATE          ERROR  PORTS
    hwp14enb23gz  test.1  nginx:latest  node1       Running        Running 6 seconds ago         
    t6mpq6sbongg  test.2  nginx:latest  controller  Running        Running 6 seconds ago      

9.4、配置HTML服务（三个节点，192.168.3.201,192.168.3.202,192.168.3.204）

    cd /var/lib/docker/volumes/nginx/_data
    mkdir html
    vim conf.d/default.conf

修改如下：

    location / {
            root   /etc/nginx/html;
            index  index.html index.htm;
        }

注：这里用/etc/nginx/html路径是因为在docker里面路径是这个

我把创建的新的html当做nginx的root目录

9.4.1、管理节点

    cd /var/lib/docker/volumes/nginx/_data/html
    touch index.html

添加内容：

    
Controller

9.4.1、node1节点

    cd /var/lib/docker/volumes/nginx/_data/html
    touch index.html

添加内容：

    
Node1

9.4.2、node2节点

    cd /var/lib/docker/volumes/nginx/_data/html
    touch index.html

添加内容：

    
Node2

9.5、重启nginx服务

当前服务在node1和controller节点运行，故而只在这两个节点执行（机器环境不同，所在的节点不同，请看清自己的节点）

9.5.1、node1节点

    [root@node1 ~]# docker ps
    CONTAINER ID        IMAGE                                                                           COMMAND                  CREATED             STATUS              PORTS               NAMES
    33b4e08c57d0        nginx@sha256:d85914d547a6c92faa39ce7058bd7529baacab7e0cd4255442b04577c4d1f424   "nginx -g 'daemon ..."   4 hours ago         Up 4 hours          80/tcp              test.1.ov76pdo715cc1f24i5vfgu6si
    docker exec -ti 33b4e08c57d0 /bin/bash
    /etc/init.d/nginx reload

9.5.2、controller节点

    [root@controller _data]# docker ps
    CONTAINER ID        IMAGE                                                                           COMMAND                  CREATED             STATUS              PORTS               NAMES
    e54913d64ce9        nginx@sha256:d85914d547a6c92faa39ce7058bd7529baacab7e0cd4255442b04577c4d1f424   "nginx -g 'daemon ..."   3 days ago          Up 3 days           80/tcp              test.2.t6mpq6sbongg5rq8rd4lofp6u
    [root@controller _data]# docker exec -ti e54913d64ce9 /bin/bash
    root@e54913d64ce9:/# /etc/init.d/nginx reload

9.6、测试服务（服务是否正常，是否有负载）

注：测试过程中有时候多curl几次才会负载到另外的一个节点

    [root@mysql76 ~]# curl http://192.168.3.204:8880/
    
Node1
    [root@mysql76 ~]# curl http://192.168.3.204:8880/
    
Controller

9.7、测试集群

在节点1执行

    docker stop 33b4e08c57d0

查看集群服务

    [root@controller _data]# docker service ps test
    ID            NAME        IMAGE         NODE        DESIRED STATE  CURRENT STATE         ERROR  PORTS
    ov76pdo715cc  test.1      nginx:latest  node2       Running        Running 1 hours ago          
    qoyuk652tky2   \_ test.1  nginx:latest  node1       Shutdown       Complete 1 hours ago         
    t6mpq6sbongg  test.2      nginx:latest  controller  Running        Running 3 days ago

测试集群服务（服务是否正常，是否有负载）

注：测试过程中有时候多curl几次才会负载到另外的一个节点

    [root@mysql76 ~]# curl http://192.168.3.204:8880/
    
Node2
    [root@mysql76 ~]# curl http://192.168.3.204:8880/
    
Controller