Kubernetes集群之安装flannel组件

  • flannel网络组件,还有一个是calico,calico支持bgp
  • overlay network:覆盖网络,在基础网络上叠加的一种虚拟网络技术模式,该网络中的主机通过虚拟链路tunnmel连接起来
  • vxlan:将原数据包封装到UDP协议中,并使用基础网络的IP/mac作为外层报文头进行封装,然后在以太网二层链路上传输,到达目的地后由隧道端点解封装并将数据发送给目标地址
  • flannel:是overlay网络中的一种,也是将源数据包封装在另一种网络包里面进行路由转发和通信,目前已经支持UDP、VXLAN、aws VPS和gce路由等数据转发方式
    1.vxlan网络拓扑
    Kubernetes集群之安装flannel组件_第1张图片
    vtep可以当成docker 0 端口理解,vtep与物理网卡之间进行nat地址转换,像这种信息也会写入到etcd中
    2.集群内不同节点间容器通讯流程
    Kubernetes集群之安装flannel组件_第2张图片
    3.写入分配的子网段到etcd中,给flannel使用
    在master节点写入
    k8s/etcd/bin/etcdctl \
    --ca-file=/k8s/etcd/ssl/ca.pem \
    --cert-file=/k8s/etcd/ssl/server.pem --key-file=/k8s/etcd/ssl/server-key.pem \
    --endpoints="https://192.168.191.130:2379,https://192.168.191.131:2379,https://192.168.191.132:2379" \
    set /coreos.com/network/config '{ "network": "172.17.0.0/16", "Backend": {"Type": "vxlan"}}'
    [root@master1 /]# k8s/etcd/bin/etcdctl \
    > --ca-file=/k8s/etcd/ssl/ca.pem \
    > --cert-file=/k8s/etcd/ssl/server.pem --key-file=/k8s/etcd/ssl/server-key.pem \
    > --endpoints="https://192.168.191.130:2379,https://192.168.191.131:2379,https://192.168.191.132:2379" \
    > set /coreos.com/network/config '{ "network": "172.17.0.0/16", "Backend": {"Type": "vxlan"}}'
    { "network": "172.17.0.0/16", "Backend": {"Type": "vxlan"}}
    [root@master1 /]# 

    查看写入的信息,在其他节点也可以查看到

    /k8s/etcd/bin/etcdctl \
    --ca-file=/k8s/etcd/ssl/ca.pem \
    --cert-file=/k8s/etcd/ssl/server.pem --key-file=/k8s/etcd/ssl/server-key.pem \
    --endpoints="https://192.168.191.130:2379,https://192.168.191.131:2379,https://192.168.191.132:2379" \
    get /coreos.com/network/config

    4.导入二进制包,flannel安装在node节点上
    哪个节点需要跑业务,哪个节点就要安装fannel组件

    [root@master1 /]# cp /abc/k8s/flannel-v0.10.0-linux-amd64.tar.gz /root/k8s/
    [root@master1 /]# cp /abc/k8s/flannel-v0.10.0-linux-amd64.tar.gz /root/k8s/
    [root@master1 /]# cd /root/k8s
    [root@master1 k8s]# scp flannel-v0.10.0-linux-amd64.tar.gz [email protected]:/opt/
    [email protected]'s password: 
    flannel-v0.10.0-linux-amd64.tar.gz                                                    100% 9479KB  53.4MB/s   00:00    
    [root@master1 k8s]# scp flannel-v0.10.0-linux-amd64.tar.gz [email protected]:/opt/
    [email protected]'s password: 
    flannel-v0.10.0-linux-amd64.tar.gz 

    5.部署与配置flannel,编辑flannel启动脚本,加入到systemd中
    以node1节点为例

    [root@node01 yum.repos.d]# cd /opt
    [root@node01 opt]# tar xf flannel-v0.10.0-linux-amd64.tar.gz 
    [root@node01 opt]# ls
    containerd  flanneld  flannel-v0.10.0-linux-amd64.tar.gz  mk-docker-opts.sh  README.md  rh

    创建fannel工作目录

    [root@node01 opt]# mkdir /k8s/flannel/{cfg,bin,ssl} -p
    [root@node01 opt]# mv mk-docker-opts.sh /k8s/flannel/bin/
    [root@node01 opt]# mv flanneld /k8s/flannel/bin/

    fannel组件启动脚本

    [root@node01 opt]# vim flannel.sh
    #!/bin/bash
    ETCD_ENDPOINTS=${1:-"http://127.0.0.1:2379"}
    cat </k8s/flannel/cfg/flanneld
    FLANNEL_OPTIONS="--etcd-endpoints=${ETCD_ENDPOINTS} \
    -etcd-cafile=/k8s/etcd/ssl/ca.pem \
    -etcd-certfile=/k8s/etcd/ssl/server.pem \
    -etcd-keyfile=/k8s/etcd/ssl/server-key.pem"
    EOF
    cat </usr/lib/systemd/system/flanneld.service
    [Unit]
    Description=Flanneld overlay address etcd agent
    After=network-online.target network.target
    Before=docker.service
    [Service]
    Type=notify
    EnvironmentFile=/k8s/flannel/cfg/flanneld
    ExecStart=/k8s/flannel/bin/flanneld --ip-masq \$FLANNEL_OPTIONS
    ExecStartPost=/k8s/flannel/bin/mk-docker-opts.sh -k DOCKER_NETWORK_OPTIONS -d /run/flannel/subnet.env
    Restart=on-failure
    [Install]
    WantedBy=multi-user.target
    EOF
    systemctl daemon-reload
    systemctl enable flanneld
    systemctl restart flanneld

    开启flannel网络功能,指定etcdIP:端口

    [root@node01 flannel]# bash flannel.sh https://192.168.191.130:2379,https://192.168.191.131:2379,https://192.168.191.132:2379

    两个node节点都需要
    6.配置docker,以使用flannel生成的子网
    以node1为例
    让docker连接flannel的网段

    [root@node01 flannel]# vim /usr/lib/systemd/system/docker.service 
    #在第十三行注释下添加
    14 EnvironmentFile=/run/flannel/subnet.env
    #在ExecStart中添加
    15 ExecStart=/usr/bin/dockerd $DOCKER_NETWORK_OPTIONS -H fd:// --containerd=/run/containerd/containerd.sock

    重启docker服务

    [root@node01 flannel]# systemctl daemon-reload
    [root@node01 flannel]# systemctl restart docker

    7.启动flannel
    查看node01节点分配的flannelIP地址,为172.17.45.0/24

    [root@node01 flannel]# cat /run/flannel/subnet.env 
    DOCKER_OPT_BIP="--bip=172.17.45.1/24"
    DOCKER_OPT_IPMASQ="--ip-masq=false"
    DOCKER_OPT_MTU="--mtu=1450"
    DOCKER_NETWORK_OPTIONS=" --bip=172.17.45.1/24 --ip-masq=false --mtu=1450"

    查看fannel网络

    [root@node01 flannel]# ifconfig
    docker0: flags=4099  mtu 1500
        inet 172.17.0.1  netmask 255.255.0.0  broadcast 172.17.255.255
    ens33: flags=4163  mtu 1500
        inet 192.168.191.131  netmask 255.255.255.0  broadcast 192.168.247.255
    flannel.1: flags=4163  mtu 1450
        inet 172.17.45.0  netmask 255.255.255.255  broadcast 0.0.0.0
    lo: flags=73  mtu 65536
        inet 127.0.0.1  netmask 255.0.0.0
    virbr0: flags=4099  mtu 1500
        inet 192.168.122.1  netmask 255.255.255.0  broadcast 192.168.122.255

    此时node1与node2中的容器就实现互通了
    在 两个node节点分别测试

    [root@node01 flannel]# docker run -it centos:7 /bin/bash
    Unable to find image 'centos:7' locally
    7: Pulling from library/centos
    ab5ef0e58194: Pull complete 
    Digest: sha256:4a701376d03f6b39b8c2a8f4a8e499441b0d567f9ab9d58e4991de4472fb813c
    Status: Downloaded newer image for centos:7
    [root@39f034a2f24e /]# yum install net-tools -y
    [root@node02 opt]# docker run -it centos:7 /bin/bash
    Unable to find image 'centos:7' locally
    7: Pulling from library/centos
    ab5ef0e58194: Pull complete 
    Digest: sha256:4a701376d03f6b39b8c2a8f4a8e499441b0d567f9ab9d58e4991de4472fb813c
    Status: Downloaded newer image for centos:7
    [root@fea29d0ff39b /]# yum install net-tools -y

    node1容器ip

    [root@39f034a2f24e /]# ifconfig
    eth0: flags=4163  mtu 1450
        inet 172.17.45.2  netmask 255.255.255.0  broadcast 172.17.45.255

    node2节点ping node1 容器

    [root@fea29d0ff39b /]# ifconfig
    eth0: flags=4163  mtu 1450
        inet 172.17.42.2  netmask 255.255.255.0  broadcast 172.17.42.255
    [root@fea29d0ff39b /]# ping 172.17.45.2
    PING 172.17.45.2 (172.17.45.2) 56(84) bytes of data.
    64 bytes from 172.17.45.2: icmp_seq=1 ttl=62 time=0.792 ms
    64 bytes from 172.17.45.2: icmp_seq=2 ttl=62 time=0.762 ms
    64 bytes from 172.17.45.2: icmp_seq=3 ttl=62 time=0.483 ms
    64 bytes from 172.17.45.2: icmp_seq=4 ttl=62 time=1.38 ms
    ^C
    --- 172.17.45.2 ping statistics ---
    4 packets transmitted, 4 received, 0% packet loss, time 3003ms
    rtt min/avg/max/mdev = 0.483/0.855/1.384/0.328 ms
    [root@fea29d0ff39b /]# 

    成功ping通。两个node节点容器互通,fannel组件安装部署成功!