k8s 中加入新node

今天在家实践了向k8s master 中加入新的node: test3  (主要就是安装docker, kubelet,kube-proxy)

部署规划

192.168.122.152   k8s-master

192.168.122.153   test3

---

1.关闭防火墙

$ systemctl stop firewalld

备注：必须关闭

2.关闭selinux

$ setenforce 0

3.关闭swap

$ swapoff -a    临时关闭

$ free             可以通过这个命令查看swap是否关闭了

$ vim /etc/fstab  永久关闭

#/dev/mapper/centos_k8s--master-swap swap                    swap    defaults        0 0

备注：必须关闭

4.添加主机名与IP对应的关系

$ vim /etc/hosts

 添加如下内容：

192.168.122.152     k8s-master

192.168.122.153     test3

5.将桥接的IPV4流量传递到iptables 的链

$ cat > /etc/sysctl.d/k8s.conf << EOF

net.bridge.bridge-nf-call-ip6tables = 1

net.bridge.bridge-nf-call-iptables = 1

EOF

$ sysctl --system

6.安装Docker

1）下载并安装

$ wget https://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo -O/etc/yum.repos.d/docker-ce.repo

$ yum -y install docker-ce-18.06.1.ce-3.el7

2）设置开机启动  

$ systemctl enable docker

$ systemctl start docker

3）查看Docker版本

$ docker --version

Docker version 18.06.1-ce, build e68fc7a

7.添加阿里云YUM软件源

直接执行如下命令    

cat > /etc/yum.repos.d/kubernetes.repo << EOF

[kubernetes]

name=Kubernetes

baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64

enabled=1

gpgcheck=1

repo_gpgcheck=1

gpgkey=https://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg

https://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg

EOF

8.安装kubeadm，kubelet和kubectl

在部署kubernetes时，要求master node和worker node上的版本保持一致，否则会出现版本不匹配导致奇怪的问题出现。本文将介绍如何在CentOS系统上，使用yum安装指定版本的Kubernetes。

yum -y install kubectl kubelet kubeadm

报如下的错：

Importing GPG key 0xA7317B0F:
 Userid     : "Google Cloud Packages Automatic Signing Key "
 Fingerprint: d0bc 747f d8ca f711 7500 d6fa 3746 c208 a731 7b0f
 From       : https://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg

Public key for 26d3e29e517cb0fd27fca12c02bd75ffa306bc5ce78c587d83a0242ba20588f0-kubectl-1.16.2-0.x86_64.rpm is not installed

 Failing package is: kubectl-1.16.2-0.x86_64
 GPG Keys are configured as: https://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg

原因是key 校验，下面的命令可以成功：

[root@k8s-master ~]# yum install -y kubelet kubeadm kubectl --nogpgcheck

$ systemctl enable kubelet

加入启动项; kubelet
 

9.Node节点加入集群

向集群添加新节点，执行在kubeadm init输出的kubeadm join命令：[root@k8s-master ~]# kubeadm token list
TOKEN     TTL       EXPIRES   USAGES    DESCRIPTION   EXTRA GROUPS
[root@k8s-master ~]# kubeadm token create
8r77xa.6lj7qhsiyw74f0up
复制上面命令，在node节点上执行

[root@k8s-node ~]# kubeadm join 192.168.122.152:6443 --token 8r77xa.6lj7qhsiyw74f0up \

>     --discovery-token-ca-cert-hash sha256:e9205bf68357eb190c5a7fda5e782d7533c361d0298093b6f283cc6886ad0b4e

[root@k8s-node ~]# kubeadm join 192.168.122.152:6443 --token blp5os.k5jjk54o61txnree \
>     --discovery-token-ca-cert-hash sha256:e9205bf68357eb190c5a7fda5e782d7533c361d0298093b6f283cc6886ad0b4e
[preflight] Running pre-flight checks
    [WARNING IsDockerSystemdCheck]: detected "cgroupfs" as the Docker cgroup driver. The recommended driver is "systemd". Please follow the guide at https://kubernetes.io/docs/setup/cri/
[preflight] Reading configuration from the cluster...
[preflight] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -oyaml'
[kubelet-start] Downloading configuration for the kubelet from the "kubelet-config-1.16" ConfigMap in the kube-system namespace
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Activating the kubelet service
[kubelet-start] Waiting for the kubelet to perform the TLS Bootstrap...

This node has joined the cluster:
* Certificate signing request was sent to apiserver and a response was received.
* The Kubelet was informed of the new secure connection details.

Run 'kubectl get nodes' on the control-plane to see this node join the cluster.

[root@k8s-node ~]#

出现上面的就说已经成功了。

如果token忘记了，则可以通过如下操作：

1）查看token，如果token失效，则重新生成一个

$ kubeadm token list

$ kubeadm token create

2）获取ca证书sha256编码hash值

$ openssl x509 -pubkey -in /etc/kubernetes/pki/ca.crt | openssl rsa -pubin -outform der 2>/dev/null | openssl dgst -sha256 -hex | sed 's/^.* //'

3）节点加入集群

$ kubeadm reset
$ kubeadm join 192.168.122.152:6443 --token 8r77xa.6lj7qhsiyw74f0up \ --discovery-token-ca-cert-hash sha256:8b79b6461e58c07333cb2851fe74fd4374af8bbbe0bf7e040b415b86ad4fb89d

10 检查一下node status:

[root@k8s-master ~]# kubectl get nodes
NAME         STATUS     ROLES    AGE    VERSION
k8s-master   Ready      master   128d   v1.16.2
k8s-node     Ready         127d   v1.16.2
test3        NotReady     41m    v1.17.3

发现notReady, 下面来研究一下。

看了kubectl describe node 发现：

runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:docker: network plugin is not ready: cni config uninitialized, CSINodeInfo is not yet initialized

网上看了解决方法是：

在：node 节点上：/var/lib/kubelet/kubeadm-flags.env

[root@test3 kubelet]# cat kubeadm-flags.env
KUBELET_KUBEADM_ARGS="--cgroup-driver=systemd --network-plugin=cni --pod-infra-container-image=registry.aliyuncs.com/google_containers/pause:3.1"

把上面红色的部分去掉。然后restart kubelet:

systemctl start kubelet