k8s集群升级

目录

1. 部署cri-docker （所有集群节点）

2. 升级master节点

3. 升级worker节点

4. 部署containerd 

---

1. 部署cri-docker （所有集群节点）

k8s从1.24版本开始移除了dockershim，所以需要安装cri-docker插件才能使用docker

软件下载：GitHub - Mirantis/cri-dockerd: dockerd as a compliant Container Runtime Interface for Kubernetes

安装

rpm -ivh  cri-dockerd-0.3.5.20231016182601.cd730ff8-0.el7.x86_64.rpm

配置cri-docker

vim /usr/lib/systemd/system/cri-docker.service

[Service]
Type=notify
ExecStart=/usr/bin/cri-dockerd --container-runtime-endpoint fd:// --network-plugin=cni --pod-infra-container-image=reg.westos.org/k8s/pause:3.7

systemctl daemon-reload
systemctl  enable --now cri-docker
ll /var/run/cri-dockerd.sock

2. 升级master节点

首先上传镜像到harbor仓库，便于升级

docker load -i k8s-v1.24.17.tar
docker images | grep k8s
docker push reg.westos.org/k8s/kube-apiserver:v1.24.17
docker push  reg.westos.org/k8s/kube-proxy:v1.24.17
docker push reg.westos.org/k8s/kube-scheduler:v1.24.17
docker push reg.westos.org/k8s/kube-controller-manager:v1.24.17
docker push reg.westos.org/k8s/pause:3.7

升级kubeadm 执行升级

yum install -y kubeadm-1.24.17-0
kubeadm upgrade plan

修改节点套接字

kubectl edit nodes k8s1

kubeadm.alpha.kubernetes.io/cri-socket: unix:///var/run/cri-dockerd.sock

kubeadm upgrade apply v1.24.17

腾空节点

kubectl drain k8s1 --ignore-daemonsets

升级kubelet

yum install -y kubelet-1.24.17-0 kubectl-1.24.17-0

配置kubelet使用cri-docker

vim /var/lib/kubelet/kubeadm-flags.env
KUBELET_KUBEADM_ARGS="--pod-infra-container-image=reg.westos.org/k8s/pause:3.7 --container-runtime=remote --container-runtime-endpoint=unix:///var/run/cri-dockerd.sock"

重启kubelet

systemctl daemon-reload 
systemctl  restart kubelet 

解除节点保护

kubectl uncordon k8s1

完成升级

3. 升级worker节点

升级kubeadm  执行升级

yum install -y kubeadm-1.24.17-0
kubeadm upgrade node

腾空节点   #需要在master节点执行

kubectl drain k8s2 --ignore-daemonsets
kubectl drain k8s3 --ignore-daemonsets

升级kubelet

yum install -y kubelet-1.24.17-0 kubectl-1.24.17-0

配置kubelet使用cri-docker

vim /var/lib/kubelet/kubeadm-flags.env
KUBELET_KUBEADM_ARGS="--pod-infra-container-image=reg.westos.org/k8s/pause:3.7 --container-runtime=remote --container-runtime-endpoint=unix:///var/run/cri-dockerd.sock"

修改节点套接字  #需要在master节点执行

kubectl edit nodes k8s2
...
kubeadm.alpha.kubernetes.io/cri-socket: unix:///var/run/cri-dockerd.sock

重启kubelet

systemctl daemon-reload 
systemctl  restart kubelet 

解除节点保护  #需要在master节点执行

kubectl uncordon k8s2

其它节点依此类推

完成升级

4. 部署containerd 

k8s从1.24版本开始移除了dockershim,所以我们不在使用docker，选用containerd。

k8s1、k8s2、k8s3在配置前需要重置节点

kubeadm reset
kubeadm reset  --cri-socket unix:///var/run/cri-dockerd.sock
kubeadm reset  --cri-socket unix:///var/run/cri-dockerd.sock

k8s1:

k8s2,3

所有节点清除iptables规则

iptables -F
iptables -F -t nat

禁用所有节点docker和cri-docker服务

systemctl  disable --now docker
systemctl  disable --now docker.socket
systemctl  disable --now cri-docker

之前部署过docker，containerd默认已经安装

修改配置

containerd config default | tee /etc/containerd/config.toml
cd /etc/containerd/
vim config.toml
...
sandbox_image = "reg.westos.org/k8s/pause:3.7"
...
SystemdCgroup = true

修改配置文件

vim /etc/containerd/config.toml
...
[plugins."io.containerd.grpc.v1.cri".registry]
      config_path = "/etc/containerd/certs.d"
mkdir -p /etc/containerd/certs.d/docker.io

vim /etc/containerd/certs.d/docker.io/hosts.toml

server = "https://registry-1.docker.io"

[host."https://reg.westos.org"]
  capabilities = ["pull", "resolve", "push"]
  skip_verify = true

拷贝证书

mkdir -p /etc/containerd/certs.d/reg.westos.org
cp /etc/docker/certs.d/reg.westos.org/ca.crt /etc/containerd/certs.d/reg.westos.org/
systemctl  restart containerd
scp -r certs.d/ config.toml k8s2:/etc/containerd/
scp -r certs.d/ config.toml k8s3:/etc/containerd/


k8s2,3
systemctl disable --now docker cri-docker docker.socket
systemctl  enable --now containerd
crictl config runtime-endpoint unix:///run/containerd/containerd.sock

启动containerd

systemctl  enable --now containerd
systemctl  restart containerd
crictl config runtime-endpoint unix:///run/containerd/containerd.sock
crictl img
crictl pull reg.westos.org/k8s/pause:3.6

集群初始化

kubeadm init --pod-network-cidr=10.244.0.0/16 --image-repository reg.westos.org/k8s --kubernetes-version v1.24.17
kubectl apply -f kube-flannel.yml