K8S 之 Flannel网络插件安装

一、flannel的作用

1、CNI网络插件最主要的功能就是实现POD资源能够跨宿主机是进行能信

#test-nodes1主机无法ping通test-nodes2主机的pod容器
[root@test-nodes1 ~]# kubectl get pods -o wide
NAME              READY   STATUS    RESTARTS   AGE     IP           NODE                      NOMINATED NODE   READINESS GATES
nginx-ds1-qg45q   1/1     Running   0          2d12h   172.7.22.3   test-nodes2.cedarhd.com              
nginx-ds1-whnmv   1/1     Running   0          2d12h   172.7.21.3   test-nodes1.cedarhd.com              
[root@test-nodes1 ~]# ping 172.7.22.3
PING 172.7.22.3 (172.7.22.3) 56(84) bytes of data.
^C
--- 172.7.22.3 ping statistics ---
8 packets transmitted, 0 received, 100% packet loss, time 6999ms

二、flannel安装

#此操作分别需要在test-nodes1与test-nodes2上操作
[root@test-nodes1 ~]#  cd /opt/src/
[root@test-nodes1 src]# wget https://github.com/coreos/flannel/releases/download/v0.11.0/flannel-v0.11.0-linux-amd64.tar.gz
[root@test-nodes1 src]# mkdir /opt/flannel-v0.11.0
[root@test-nodes1 src]# tar xf flannel-v0.11.0-linux-amd64.tar.gz -C /opt/flannel-v0.11.0/
[root@test-nodes1 src]# ln -s /opt/flannel-v0.11.0/ /opt/flannel
[root@test-nodes1 src]# cd /opt/flannel
[root@test-nodes1 flannel]# mkdir cert
[root@test-nodes1 flannel]# cd cert/
[root@test-nodes1 cert]# scp test-operator:/opt/certs/ca.pem .
root@test-operator's password: 
ca.pem                                                                                                                      100% 1346   173.2KB/s   00:00    
[root@test-nodes1 cert]# scp test-operator:/opt/certs/client.pem .
root@test-operator's password: 
client.pem                                                                                                                  100% 1363   207.0KB/s   00:00    
[root@test-nodes1 cert]# scp test-operator:/opt/certs/client-key.pem .
root@test-operator's password: 
client-key.pem                                        
[root@test-nodes1 cert]# cd ..
flannel]# vi subnet.env
FLANNEL_NETWORK=172.7.0.0/16
FLANNEL_SUBNET=172.7.21.1/24         #每个nodes的子网不一样需修改，test-nodes1为21，而test-nodes2为22
FLANNEL_MTU=1500
FLANNEL_IPMASQ=false

flannel]# vi flanneld.sh 
#!/bin/sh
./flanneld \
  --public-ip=10.3.153.221 \               #另外一台test-nodes2需要修改
  --etcd-endpoints=https://10.3.153.212:2379,https://10.3.153.221:2379,https://10.3.153.222:2379 \
  --etcd-keyfile=./cert/client-key.pem \
  --etcd-certfile=./cert/client.pem \
  --etcd-cafile=./cert/ca.pem \
  --iface=ens33 \
  --subnet-file=./subnet.env \
  --healthz-port=2401

[root@test-nodes1 flannel]# cd /opt/etcd
# 下面这一步在一步机器上执行即可，只需执行一次
[root@test-nodes1 etcd]# ./etcdctl set /coreos.com/network/config '{"Network": "172.7.0.0/16", "Backend": {"Type": "host-gw"}}'
{"Network": "172.7.0.0/16", "Backend": {"Type": "host-gw"}}

[root@test-nodes1 etcd]# vi /etc/supervisord.d/flannel.ini
[program:flanneld-7-21]
command=/opt/flannel/flanneld.sh                             ; the program (relative uses PATH, can take args)
numprocs=1                                                   ; number of processes copies to start (def 1)
directory=/opt/flannel                                       ; directory to cwd to before exec (def no cwd)
autostart=true                                               ; start at supervisord start (default: true)
autorestart=true                                             ; retstart at unexpected quit (default: true)
startsecs=30                                                 ; number of secs prog must stay running (def. 1)
startretries=3                                               ; max # of serial start failures (default 3)
exitcodes=0,2                                                ; 'expected' exit codes for process (default 0,2)
stopsignal=QUIT                                              ; signal used to kill process (default TERM)
stopwaitsecs=10                                              ; max num secs to wait b4 SIGKILL (default 10)
user=root                                                    ; setuid to this UNIX account to run the program
redirect_stderr=true                                         ; redirect proc stderr to stdout (default false)
stdout_logfile=/data/logs/flanneld/flanneld.stdout.log       ; stderr log path, NONE for none; default AUTO
stdout_logfile_maxbytes=64MB                                 ; max # logfile bytes b4 rotation (default 50MB)
stdout_logfile_backups=4                                     ; # of stdout logfile backups (default 10)
stdout_capture_maxbytes=1MB                                  ; number of bytes in 'capturemode' (default 0)
stdout_events_enabled=false                                  ; emit events on stdout writes (default false)

[root@test-nodes1 etcd]# supervisorctl update
flanneld-7-21: added process group
[root@test-nodes1 flannel]# supervisorctl status
etcd-server-7-21                 RUNNING   pid 14828, uptime 2 days, 14:55:02
flanneld-7-21                    RUNNING   pid 26991, uptime 0:00:48
kube-apiserver-7-21              RUNNING   pid 14810, uptime 2 days, 14:56:17
kube-controller-manager-7-21     RUNNING   pid 14868, uptime 2 days, 14:46:46
kube-kubelet-7-21                RUNNING   pid 15095, uptime 2 days, 13:46:15
kube-proxy-7-21                  RUNNING   pid 22013, uptime 2 days, 13:05:18
kube-scheduler-7-21              RUNNING   pid 25120, uptime 2 days, 12:53:29

三、flannel 工作原理

1、host-gw模型，两台运算节点在同一个网段，当flannel启动时会为两台运算节点增加相应的路由表，此时两台运算节点的POD就能互相通信，而flannel退出也不会影响，因为路由表已添加到宿主机。

2、VxLAN模型，两台运算节点在不同的网段