k8s集群 后续追加新节点

k8s1.20.1 集群安装 搭建完毕之后，从2020-12-30到2021-1-11的10多天时间里，突然想再给k8s集群增加个节点。

1. knode2节点完全克隆一个knode3

1. 1 机器环境

节点hostname	作用	IP
kmaster	master	192.168.8.121
knode1	node1	192.168.8.122
knode2	node2	192.168.8.123
knode3	node3	192.168.8.124

1.2 hostname

[root@knode2 ~]# hostnamectl set-hostname knode3 --static

1.3 网络设置

[root@knode2 ~]# vi /etc/sysconfig/network-scripts/ifcfg-ens33
BOOTPROTO="static" #dhcp改为static 
ONBOOT="yes" #开机启用本配置
IPADDR=192.168.8.121 #静态IP 192.168.8.122/192.168.8.123
GATEWAY=192.168.8.2 #默认网关
NETMASK=255.255.255.0 #子网掩码
DNS1=114.114.114.114 #DNS 配置
DNS2=8.8.8.8 #DNS 配置

[root@knode2 ~]# systemctl restart network

1.4 配置IP host映射关系

[root@knode2 ~]# echo "192.168.8.124 knode3" >> /etc/hosts
[root@knode2 ~]# reboot

2 往kmaster节点追加工作节点

2.1 查看追加命令（在kmaster查看）·

[root@kmaster ~]# cat kubeadm-init.log
Then you can join any number of worker nodes by running the following on each as root:

kubeadm join 192.168.8.121:6443 --token abcdef.0123456789abcdef \
    --discovery-token-ca-cert-hash sha256:7459fa01464531734d3eee182461b77b043d31eff7df2233635654d7c199c947

2.2 删除从knode2克隆的knode3连接信息

[root@knode3 ~]# rm -f /etc/kubernetes/kubelet.conf
[root@knode3 ~]# rm -f /etc/kubernetes/pki/ca.crt

2.3 追加节点(在knode3执行)

这里可能执行的时间稍稍有点长

[root@knode3 ~]# kkubeadm join 192.168.8.121:6443 --token abcdef.0123456789abcdef \
    --discovery-token-ca-cert-hash sha256:7459fa01464531734d3eee182461b77b043d31eff7df2233635654d7c199c947
[preflight] Running pre-flight checks
        [WARNING SystemVerification]: this Docker version is not on the list of validated versions: 20.10.1. Latest validated version: 19.03
error execution phase preflight: couldn't validate the identity of the API Server: could not find a JWS signature in the cluster-info ConfigMap for token ID "abcdef"
To see the stack trace of this error execute with --v=5 or higher

2.4 解决问题：

查看认证信息（在看master查看）--原有的已经截止到12-31过期了

[root@kmaster ~]# kubeadm token list
TOKEN                     TTL         EXPIRES                     USAGES                   DESCRIPTION                                                EXTRA GROUPS
abcdef.0123456789abcdef      2020-12-31T09:44:48+09:00   authentication,signing                                                        system:bootstrappers:kubeadm:default-node-token

[root@kmaster ~]# 

造成这种问题的原因可能有两点：token失效 、CA证书失效。

2.4.1 生成token

生成永久token

[root@kmaster ~]# kubeadm token create --ttl 0
oij2et.1wl9wap9p2wp7at9

生成新的token:查看后有效期只有23小时

[root@kmaster ~]# kubeadm token create
m3nr37.81bu8er3ibyoyvw7
[root@kmaster ~]# kubeadm token list
TOKEN                     TTL         EXPIRES                     USAGES                   DESCRIPTION                                                EXTRA GROUPS
m3nr37.81bu8er3ibyoyvw7   23h         2021-01-12T23:29:49+09:00   authentication,signing                                                        system:bootstrappers:kubeadm:default-node-token

2.4.2 查看crt

[root@kmaster ~]# openssl x509 -pubkey -in /etc/kubernetes/pki/ca.crt | openssl rsa -pubin -outform der 2>/dev/null | openssl dgst -sha256 -hex | sed 's/^.* //'
7459fa01464531734d3eee182461b77b043d31eff7df2233635654d7c199c947

2.4.3 把上述2.4.1和2.4.2组装后，再次试用

追加节点(在knode3执行)

[root@knode3 ~]# systemctl stop kubelet
[root@knode3 ~]# rm -f /etc/kubernetes/kubelet.conf
[root@knode3 ~]# rm -f /etc/kubernetes/pki/ca.crt
[root@knode3 ~]# systemctl start kubelet
[root@knode3 ~]# kubeadm join 192.168.8.121:6443 --token oij2et.1wl9wap9p2wp7at9 \
    --discovery-token-ca-cert-hash sha256:7459fa01464531734d3eee182461b77b043d31eff7df2233635654d7c199c947
...
error execution phase kubelet-start: error uploading crisocket: timed out waiting for the condition

2.4.4 错误解决（...timed out waiting for the condition）

[root@knode3 ~]# kubeadm reset

再次重复2.4.3

3 查看追加后的集群

[root@kmaster ~]# kubectl get node
NAME      STATUS     ROLES                  AGE   VERSION
kmaster   Ready      control-plane,master   12d   v1.20.1
knode1    Ready                       12d   v1.20.1
knode2    NotReady                    12d   v1.20.1
knode3    Ready                       55s   v1.20.1