【k8s系列】vmware中ubuntu22.04使用kubeadm安装k8s集群(1.24.4 or higher)(cka考试环境搭建)

author: ningan123
date: ‘2022-11-28 22:53’
updated: ‘2022-12-17 11:30’

搭建虚拟机可以参考我的另一篇博文：【网络成长记】VMware16.2.2中创建ubuntu22.04.1虚拟机+修改静态ip+换源+mobaxterm远程连接+修改命令提示符颜色(cka考试虚拟机搭建)

部署环境准备

ubuntu22.04虚拟机3台

环境修改

修改hostname(各修改各的)

hostnamectl set-hostname k8s-master // master节点
hostnamectl set-hostname k8s-node1  // node1节点
hostnamectl set-hostname k8s-node2  // node2节点

重启虚拟机，可以看到hostname已经生效了

修改hosts文件(master 和所有node )

修改/etc/hosts文件，添加下面内容

cat >> /etc/hosts << EOF
192.168.1.100 k8s-master
192.168.1.101 k8s-node1
192.168.1.102 k8s-node2
EOF

禁用swap(master 和所有node)

执行 free -m 命令检测: 如果Swap这一行不是0，则说明虚拟内存swap被开启了，需要关闭。

sudo swapoff -a 
sudo sed -i '/swap/ s/^\(.*\)$/#\1/g' /etc/fstab 
# -i 插入修改模式，g 标识符表示全局查找替换,表示注释掉swap的那一行。 free -h

修改内核参数(master 和所有node )

# Enable kernel modules
sudo tee /etc/modules-load.d/containerd.conf <<EOF
overlay
br_netfilter
EOF
sudo modprobe overlay
sudo modprobe br_netfilter



# Add some settings to sysctl
sudo tee /etc/sysctl.d/kubernetes.conf<<EOF
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
net.ipv4.ip_forward = 1
EOF

# 重新加载 sysctl
sudo sysctl --system

安装 containerd

# 安装关联软件
sudo apt install -y curl gnupg2 software-properties-common apt-transport-https ca-certificates

# 安装docker仓库
sudo curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmour -o /etc/apt/trusted.gpg.d/docker.gpg
sudo add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable"

# 安装containerd
sudo apt update 
sudo apt install -y containerd.io

# 配置 containerd 用systemdcgroup启动.
containerd config default | sudo tee /etc/containerd/config.toml >/dev/null 2>&1
sudo sed -i 's/SystemdCgroup \= false/SystemdCgroup \= true/g' /etc/containerd/config.toml

# 重启
sudo systemctl restart containerd 
sudo systemctl enable containerd

可以看到containerd已经成功启动~

# 设置crictl
cat > /etc/crictl.yaml << EOF
runtime-endpoint: unix:///var/run/containerd/containerd.sock
image-endpoint: unix:///var/run/containerd/containerd.sock
timeout: 10 
debug: false
EOF

安装k8s

安装组件(master 和所有node )

# 添加apt repository for Kubernetes
curl https://mirrors.aliyun.com/kubernetes/apt/doc/apt-key.gpg | sudo apt-key add - 
sudo apt-add-repository "deb https://mirrors.aliyun.com/kubernetes/apt/ kubernetes-xenial main"

apt-get update

# 查看版本
apt-cache madison kubeadm|head

apt install -y kubelet=1.24.4-00 kubeadm=1.24.4-00 kubectl=1.24.4-00

下载镜像(master 和所有node )

# 使用国内阿里云镜像站点，查看所需镜像
kubeadm config images list \
--image-repository registry.aliyuncs.com/google_containers \
--kubernetes-version=v1.24.4

# 指定版本下载
kubeadm config images pull \
--kubernetes-version=v1.24.4 \
--image-repository registry.aliyuncs.com/google_containers

# 查看镜像
crictl images

集群配置(master)

kubeadm config print init-defaults > kubeadm.yaml

修改集群配置如下：
左边为导出的原始文件，右边为修改后的文件

# 
kubeadm init \
--config /root/kubeadm.yaml \
--ignore-preflight-errors=SystemVerification \
--upload-certs

 journalctl -xefu  kubelet

集群配置2

kubeadm reset

集群配置3

集群配置4

journalctl -xefu kubelet 的报错信息如下：

11月 28 23:24:10 k8s-master kubelet[24909]: E1128 23:24:10.831329   24909 remote_runtime.go:201] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to get sandbox image \"registry.k8s.io/pause:3.6\": failed to pull image \"registry.k8s.io/pause:3.6\": failed to pull and unpack image \"registry.k8s.io/pause:3.6\": failed to resolve reference \"registry.k8s.io/pause:3.6\": failed to do request: Head \"https://asia-east1-docker.pkg.dev/v2/k8s-artifacts-prod/images/pause/manifests/3.6\": dial tcp 108.177.125.82:443: connect: connection refused"

cat /etc/containerd/config.toml  |grep image

# 替换
sed -i "s#registry.k8s.io/pause:3.6#registry.aliyuncs.com/google_containers/pause:3.6#g"  /etc/containerd/config.toml

# 重启containerd
systemctl daemon-reload && systemctl restart containerd

# 重新执行
kubeadm reset

kubeadm init \
--config /root/kubeadm.yaml \
--ignore-preflight-errors=SystemVerification \
--upload-certs

集群配置5

journalctl -xefu kubelet 的报错信息如下：

11月 29 00:11:19 k8s-master kubelet[42857]: E1129 00:11:19.795996   42857 remote_runtime.go:201] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to create containerd task: failed to create shim task: OCI runtime create failed: runc create failed: expected cgroupsPath to be of format \"slice:prefix:name\" for systemd cgroups, got \"/kubepods/burstable/pod444f719af02c349e8eca993c949ab6b9/eddc1ba340a885d45775c0bf4e1f634abf90ac1474ffea30ea6489ba5edd96ba\" instead: unknown"

改回systemd

# 重新执行
kubeadm reset

kubeadm init \
--config /root/kubeadm.yaml \
--ignore-preflight-errors=SystemVerification \
--upload-certs

  mkdir -p $HOME/.kube
  sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
  sudo chown $(id -u):$(id -g) $HOME/.kube/config

# 最终配置
[root@k8s-master ~]# cat kubeadm.yaml
apiVersion: kubeadm.k8s.io/v1beta3
bootstrapTokens:
- groups:
  - system:bootstrappers:kubeadm:default-node-token
  token: abcdef.0123456789abcdef
  ttl: 24h0m0s
  usages:
  - signing
  - authentication
kind: InitConfiguration
localAPIEndpoint:
  advertiseAddress: 192.168.1.100
  bindPort: 6443
nodeRegistration:
  criSocket: unix:///var/run/containerd/containerd.sock
  imagePullPolicy: IfNotPresent
  name: k8s-master
  taints: null
---
apiServer:
  timeoutForControlPlane: 4m0s
  certSANs:
  - 192.168.1.100
apiVersion: kubeadm.k8s.io/v1beta3
certificatesDir: /etc/kubernetes/pki
clusterName: kubernetes
controllerManager: {}
dns: {}
etcd:
  local:
    dataDir: /var/lib/etcd
imageRepository: registry.cn-hangzhou.aliyuncs.com/google_containers
kind: ClusterConfiguration
kubernetesVersion: 1.24.4
networking:
  dnsDomain: cluster.local
  podSubnet: 10.244.0.0/16
  serviceSubnet: 10.96.0.0/12
scheduler: {}
---
# 指定cgroup
apiVersion: kubelet.config.k8s.io/v1beta1
kind: KubeletConfiguration
cgroupDriver: systemd

node节点

两个节点都执行如下方法：

cat /etc/containerd/config.toml  |grep image

sed -i "s#registry.k8s.io/pause:3.6#registry.aliyuncs.com/google_containers/pause:3.6#g"  /etc/containerd/config.toml

systemctl daemon-reload && systemctl restart containerd

kubeadm join 192.168.1.100:6443 --token abcdef.0123456789abcdef \
        --discovery-token-ca-cert-hash sha256:3ce5d8f9fd07a09dd1625531de6876f3b41110c4e005a1c2b5470fbf074ab5cf

集群安装成功~

网络插件

# 使用calico
curl https://raw.githubusercontent.com/projectcalico/calico/v3.24.1/manifests/calico.yaml -o calico-3-24-1.yaml
kubectl apply -f calico-3-24-1.yaml

部署完成之后多等一会，就会变成running了~

node也全部都是ready了~

测试

kubectl create deployment nginx --image=nginx
kubectl get pod
kubectl expose deployment nginx --port=80 --type=NodePort
kubectl get pod,svc

工具类学习

grep 'registry.k8s.io/pause:3.6' /var/log/syslog

参考

# [最新]ubuntu22.04安装kubernetes1.25 k8s1.25 前半部分配置
Ubuntu22.04 安装 K8S 后半部分配置 kubeadm的config配置 ipvs配置