kubeadm部署指定版本的K8s+containerd+docker——图文详细版（上）
Containerd的起源与发展
Docker 强势崛起，云计算开始容器时代，Dockers以独特的容器架构和容器“镜像”快速发展，对其他容器技术进行致命的降维打击，包括 Google在内的很多公司无法与之匹敌。Google和其它互联网公司为了不被Docker占领全部市场，与 Docker 公司联合推进一个开源的容器运行时作为 Docker 的核心依赖——Containerd，Containerd 是一个工业级标准的容器运行时，它强调简单性、健壮性和可移植性。其诞生于Docker，提供如下功能：
1、管理容器的生命周期（从创建容器到销毁容器）
2、拉取/推送容器镜像
3、存储管理（管理镜像及容器数据的存储）
4、调用 runc 运行容器（与 runc 等容器运行时交互）
5、管理容器网络接口及网络
Containerd架构如下图所示
而后，Google 联合 Red Hat等与 Docker 公司商讨将libcontainer捐给中立的社区（OCI，Open Container Intiative），并改名为Runc。Docker公司退役后，Google 等又合伙成立了CNCF（Cloud Native Computing Fundation）进行大规模容器编排，以此与Docker抗衡。Docker 公司推出了Swarm与Kubernetes进行抗衡，但结果一目了然。
Kubernetes 设计了一套接口规则CRI（Container Runntime Interface），第一个支持该接口规则的是Containerd。为继续支持Docker，专门组件中集成了一个shim，其可以将 CRI 调用翻译成 Docker 的 API，以此支持Docker使用。本文通过Kubeadm部署指定版本的Kubernetes，并同时安装Containerd+Docker，支持两种容器运行时。本文参考Containerd 使用教程和kubeadm部署K8S集群并使用containerd做容器运行时。
安装完成后，接下来的文章会汇总K8s基本操作命令、Containerd基本操作命令和Dockers基本操作命令，欢迎大家以前讨论。
下面我们将实战安装一下kubernetes+Containerd+Docker
- 本次安装主要是用来学习kubernetes指定版本安装的相关过程及组件的配置，若用于企业实践可根据具体需要在节点添加相应的组件
- 前面文章给出了如何在物理机上安装centos操作系统——具体可参考安装配置centos7虚拟机（详精版）安装虚拟机，或者参考物理机安装centos7(u盘安装）——详细版配置物理机， 参考MobaXterm连接局域网的虚拟机连接虚拟机，也可直接操作虚拟机。（小白可直接参考本文内容直接完成K8s+Containerd+Docker的安装，需要改的参数会具体指出）
- 如果是新安装的操作系统，可以安装下列组件，方便操作（其它可跳过第一阶段，直接看准备工作）——具体参考使用二进制安装包部署kubernetes（详细精讲版）中安装工具部分
- 放在bash输入框中的都是命令，可复制直接使用
- 虚拟机上安装可准备2台centos系统的虚拟机，一台master,一台node
- 本文使用两台虚拟机，一台即是master也是node（通过taint实现），一台node
- 本文使用的系统是centos7，内核版本为5.4.207-1.el7.elrepo.x86_64，可参考Centos7升级内核——图文详尽版升级内核，内存1G以上，硬盘20G以上，在物理机或虚拟机安装皆可
- 安装如果出现错误，我会把遇到的问题汇总在文章尾部，大家可以参考本文安装过程中遇到的问题解决相关问题。
总体规划
| 主机名 | IP地址 | 角色 |
|---|---|---|
| master | 192.168.110.129 | master |
| node1 | 192.168.110.130 | node1 |
本文master和node1 组件说明
| 角色 | 组件 |
| master | docker、containerd、kubelet、kubeadm、kubectl |
| node1 | docker、containerd、kubelet、kubeadm、kubectl |
1、准备工作
本文使用的主机IP为192.168.110.129（master）和192.168.110.130(node1),请根据自己主机IP修改为对应地址，文章后续不再强调。每个bash代码框（运行需要的命令）后会给出本文实践的代码效果（一般会有root和主机名），可对比看运行的有没有问题。
1）设置主机名（本文用的是root用户，非root用户可加入sudo命令）（可跳过该步骤，该步骤可需改主机名称为master和node1）
在主机master（192.168.110.129）修改主机名
hostnamectl set-hostname master在主机node1（192.168.110.130）修改主机名
hostnamectl set-hostname node12)修改 /etc/hostname 文件，添加主机名和 IP 的对应关系：(master（192.168.110.129）,node1（192.168.110.130）)(两台都需要修改)
vim /etc/hostsi进入编辑状态，修改/etc/hosts为如下（修改IP为自己主机的IP，后续不再说明）
[root@localhost ~]# cat /etc/hosts
192.168.110.129 master
192.168.110.130 node1:wq保存并退出
reboot重启后主机名就改为master和node1,同时可以通过master和node1访问192.168.110.129和192.168.110.130。
3）关闭防火墙，关闭服务，并设为开机不自启，并清空防火墙规则（两台都需要）
sudo systemctl stop firewalld && sudo systemctl disable firewalld查看防火墙的状态，为关闭状态
systemctl status firewalld[root@master ~]# systemctl stop firewalld && sudo systemctl disable firewalld
Removed symlink /etc/systemd/system/multi-user.target.wants/firewalld.service.
Removed symlink /etc/systemd/system/dbus-org.fedoraproject.FirewallD1.service.
[root@master ~]# systemctl status firewalld
● firewalld.service - firewalld - dynamic firewall daemon
Loaded: loaded (/usr/lib/systemd/system/firewalld.service; disabled; vendor preset: enabled)
Active: inactive (dead)
Docs: man:firewalld(1)
7月 28 20:03:32 master systemd[1]: Starting firewalld - dynamic firewall daemon...
7月 28 20:03:33 master systemd[1]: Started firewalld - dynamic firewall daemon.
7月 28 20:03:33 master firewalld[875]: WARNING: AllowZoneDrifting is enabled. This is considered an insecure configu...t now.
7月 28 20:10:41 master systemd[1]: Stopping firewalld - dynamic firewall daemon...
7月 28 20:10:41 master systemd[1]: Stopped firewalld - dynamic firewall daemon.
Hint: Some lines were ellipsized, use -l to show in full.sudo iptables -F && sudo iptables -X && sudo iptables -F -t nat && sudo iptables -X -t nat
sudo iptables -P FORWARD ACCEPT4）如果开启了 swap 分区，kubelet 会启动失败(可以通过将参数 --fail-swap-on 设置为false 来忽略 swap on)，故需要在每台机器上关闭 swap 分区（两台都需要）
sudo swapoff -a为了防止开机自动挂载 swap 分区，可以注释 /etc/fstab 中相应的条目：
sudo sed -i '/ swap / s/^\(.*\)$/#\1/g' /etc/fstab5）关闭 SELinux，否则后续 K8S 挂载目录时可能报错 Permission denied （两台都需要）
sudo setenforce 0修改配置文件，永久生效；
vi /etc/selinux/config修改SELINUX=disabled，:wq保存并退出
查询/etc/selinux/config
grep SELINUX /etc/selinux/config
SELINUX=disabled本文执行效果
[root@master ~]# sudo iptables -F && sudo iptables -X && sudo iptables -F -t nat && sudo iptables -X -t nat
[root@master ~]# sudo iptables -P FORWARD ACCEPT
[root@master ~]# sudo swapoff -a
[root@master ~]# sudo sed -i '/ swap / s/^\(.*\)$/#\1/g' /etc/fstab
[root@master ~]# sudo setenforce 0
[root@master ~]# vi /etc/selinux/config
[root@master ~]# grep SELINUX /etc/selinux/config
# SELINUX= can take one of these three values:
SELINUX=disabled
# SELINUXTYPE= can take one of three values:
SELINUXTYPE=targeted6）时间同步 （两台都需要）
yum install ntpdate -y && ntpdate time.windows.com7）配置内核参数 （两台都需要）
cat > /etc/sysctl.d/k8s.conf <[root@master ~]# yum install ntpdate -y && ntpdate time.windows.com
已加载插件：fastestmirror, langpacks
Loading mirror speeds from cached hostfile
* base: mirrors.aliyun.com
* elrepo: mirrors.tuna.tsinghua.edu.cn
* extras: mirrors.ustc.edu.cn
* updates: mirrors.ustc.edu.cn
base | 3.6 kB 00:00:00
elrepo | 3.0 kB 00:00:00
extras | 2.9 kB 00:00:00
mysql-connectors-community | 2.6 kB 00:00:00
mysql-tools-community | 2.6 kB 00:00:00
mysql57-community | 2.6 kB 00:00:00
updates | 2.9 kB 00:00:00
(1/3): mysql-tools-community/x86_64/primary_db | 87 kB 00:00:00
(2/3): mysql-connectors-community/x86_64/primary_db | 90 kB 00:00:00
(3/3): mysql57-community/x86_64/primary_db | 315 kB 00:00:00
软件包 ntpdate-4.2.6p5-29.el7.centos.2.x86_64 已安装并且是最新版本
无须任何处理
28 Jul 12:25:30 ntpdate[3081]: step time server 20.189.79.72 offset -28799.842816 sec
[root@master ~]# cat > /etc/sysctl.d/k8s.conf < net.bridge.bridge-nf-call-ip6tables = 1
> net.bridge.bridge-nf-call-iptables = 1
> EOF
[root@master ~]# sysctl --system
* Applying /usr/lib/sysctl.d/00-system.conf ...
* Applying /usr/lib/sysctl.d/10-default-yama-scope.conf ...
* Applying /usr/lib/sysctl.d/50-default.conf ...
kernel.sysrq = 16
kernel.core_uses_pid = 1
kernel.kptr_restrict = 1
net.ipv4.conf.default.rp_filter = 1
net.ipv4.conf.all.rp_filter = 1
net.ipv4.conf.default.accept_source_route = 0
net.ipv4.conf.all.accept_source_route = 0
net.ipv4.conf.default.promote_secondaries = 1
net.ipv4.conf.all.promote_secondaries = 1
fs.protected_hardlinks = 1
fs.protected_symlinks = 1
* Applying /usr/lib/sysctl.d/60-libvirtd.conf ...
fs.aio-max-nr = 1048576
* Applying /etc/sysctl.d/99-sysctl.conf ...
* Applying /etc/sysctl.d/k8s.conf ...
* Applying /etc/sysctl.conf ... 8）配置br_netfilter，不配置初始化易出错
modprobe br_netfilter
echo 1 > /proc/sys/net/bridge/bridge-nf-call-iptables
echo 1 > /proc/sys/net/ipv4/ip_forward准备工作完成！
2、安装Containerd（两台都需要）
安装Containerd环境依赖并设置yum源并查看Containerd版本
yum install -y yum-utils device-mapper-persistent-data lvm2
yum-config-manager --add-repo http://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo
yum list | grep containerd[root@node1 ~]# yum install -y yum-utils device-mapper-persistent-data lvm2
已加载插件：fastestmirror, langpacks
Loading mirror speeds from cached hostfile
* base: mirrors.tuna.tsinghua.edu.cn
* elrepo: mirrors.tuna.tsinghua.edu.cn
* extras: ftp.sjtu.edu.cn
* updates: ftp.sjtu.edu.cn
base | 3.6 kB 00:00:00
elrepo | 3.0 kB 00:00:00
extras | 2.9 kB 00:00:00
updates | 2.9 kB 00:00:00
(1/2): elrepo/primary_db | 396 kB 00:00:02
(2/2): updates/7/x86_64/primary_db | 16 MB 00:00:21
软件包 yum-utils-1.1.31-54.el7_8.noarch 已安装并且是最新版本
软件包 device-mapper-persistent-data-0.8.5-3.el7_9.2.x86_64 已安装并且是最新版本
软件包 7:lvm2-2.02.187-6.el7_9.5.x86_64 已安装并且是最新版本
无须任何处理
[root@node1 ~]# yum-config-manager --add-repo http://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo
已加载插件：fastestmirror, langpacks
adding repo from: http://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo
grabbing file http://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo to /etc/yum.repos.d/docker-ce.repo
repo saved to /etc/yum.repos.d/docker-ce.repo
[root@node1 ~]# yum list | grep containerd
containerd.io.x86_64 1.6.6-3.1.el7 docker-ce-stable选择安装Containerd的版本，如containerd.io-1.6.6-3.1.el7（7如上图查询结果中出现的版本皆可选择，本文containerd.io-1.6.6-3.1.el17），默认版本安装使用如下命名
yum -y install containerd指定版本安装,这里以containerd.io-1.6.6-3.1.el17为例
yum -y install containerd.io-1.6.6-3.1.el17[root@node1 ~]# yum -y install containerd
已加载插件：fastestmirror, langpacks
Loading mirror speeds from cached hostfile
* base: mirrors.tuna.tsinghua.edu.cn
* elrepo: mirrors.tuna.tsinghua.edu.cn
* extras: ftp.sjtu.edu.cn
* updates: ftp.sjtu.edu.cn
正在解决依赖关系
There are unfinished transactions remaining. You might consider running yum-complete-transaction, or "yum-complete-transaction --cleanup-only" and "yum history redo last", first to finish them. If those don't work you'll have to try removing/installing packages by hand (maybe package-cleanup can help).
--> 正在检查事务
---> 软件包 containerd.io.x86_64.0.1.6.6-3.1.el7 将被 安装
--> 正在处理依赖关系 container-selinux >= 2:2.74，它被软件包 containerd.io-1.6.6-3.1.el7.x86_64 需要
--> 正在检查事务
---> 软件包 container-selinux.noarch.2.2.119.2-1.911c772.el7_8 将被 安装
--> 解决依赖关系完成
依赖关系解决
==============================================================================================================================
Package 架构 版本 源 大小
==============================================================================================================================
正在安装:
containerd.io x86_64 1.6.6-3.1.el7 docker-ce-stable 33 M
为依赖而安装:
container-selinux noarch 2:2.119.2-1.911c772.el7_8 extras 40 k
事务概要
==============================================================================================================================
安装 1 软件包 (+1 依赖软件包)
总下载量：33 M
安装大小：125 M
Downloading packages:
(1/2): container-selinux-2.119.2-1.911c772.el7_8.noarch.rpm | 40 kB 00:00:00
warning: /var/cache/yum/x86_64/7/docker-ce-stable/packages/containerd.io-1.6.6-3.1.el7.x86_64.rpm: Header V4 RSA/SHA512 Signature, key ID 621e9f35: NOKEY
containerd.io-1.6.6-3.1.el7.x86_64.rpm 的公钥尚未安装
(2/2): containerd.io-1.6.6-3.1.el7.x86_64.rpm | 33 MB 00:00:44
------------------------------------------------------------------------------------------------------------------------------
总计 760 kB/s | 33 MB 00:00:44
从 https://mirrors.aliyun.com/docker-ce/linux/centos/gpg 检索密钥
导入 GPG key 0x621E9F35:
用户ID : "Docker Release (CE rpm) "
指纹 : 060a 61c5 1b55 8a7f 742b 77aa c52f eb6b 621e 9f35
来自 : https://mirrors.aliyun.com/docker-ce/linux/centos/gpg
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
正在安装 : 2:container-selinux-2.119.2-1.911c772.el7_8.noarch 1/2
正在安装 : containerd.io-1.6.6-3.1.el7.x86_64 2/2
验证中 : containerd.io-1.6.6-3.1.el7.x86_64 1/2
验证中 : 2:container-selinux-2.119.2-1.911c772.el7_8.noarch 2/2
已安装:
containerd.io.x86_64 0:1.6.6-3.1.el7
作为依赖被安装:
container-selinux.noarch 2:2.119.2-1.911c772.el7_8
完毕！ 创建Containerd的配置文件并修改相应配置
Containerd 的默认配置文件为/etc/containerd/config.toml，我们可以通过命令来生成一个默认的配置,，需要把Containerd相关的文件都放入/etc/containerd文件夹，创建/etc/containerd文件夹并生成Containerd的配置文件。
mkdir -p /etc/containerd
containerd config default > /etc/containerd/config.toml在国内拉取公共镜像仓库的速度比较慢，为了节约拉取时间，需要为 Containerd 配置镜像仓库的mirro，其中Containerd和Docker相比的区别（来源于文章Containerd 使用教程）
- Containerd 只支持通过CRI拉取镜像的mirro，也就是说，只有通过crictl或者K8s调用时mirro 才会生效，通过ctr拉取是不会生效的。
- Docker只支持为Docker Hub配置mirror，而Containerd支持为任意镜像仓库配置mirror。
配置文件具体如下，需要修改配置文件中的registry 配置块
[root@node1 ~]# cat /etc/containerd/config.toml
disabled_plugins = []
imports = []
oom_score = 0
plugin_dir = ""
required_plugins = []
root = "/var/lib/containerd"
state = "/run/containerd"
temp = ""
version = 2
[cgroup]
path = ""
[debug]
address = ""
format = ""
gid = 0
level = ""
uid = 0
[grpc]
address = "/run/containerd/containerd.sock"
gid = 0
max_recv_message_size = 16777216
max_send_message_size = 16777216
tcp_address = ""
tcp_tls_ca = ""
tcp_tls_cert = ""
tcp_tls_key = ""
uid = 0
[metrics]
address = ""
grpc_histogram = false
[plugins]
[plugins."io.containerd.gc.v1.scheduler"]
deletion_threshold = 0
mutation_threshold = 100
pause_threshold = 0.02
schedule_delay = "0s"
startup_delay = "100ms"
[plugins."io.containerd.grpc.v1.cri"]
device_ownership_from_security_context = false
disable_apparmor = false
disable_cgroup = false
disable_hugetlb_controller = true
disable_proc_mount = false
disable_tcp_service = true
enable_selinux = false
enable_tls_streaming = false
enable_unprivileged_icmp = false
enable_unprivileged_ports = false
ignore_image_defined_volumes = false
max_concurrent_downloads = 3
max_container_log_line_size = 16384
netns_mounts_under_state_dir = false
restrict_oom_score_adj = false
sandbox_image = "k8s.gcr.io/pause:3.6"
selinux_category_range = 1024
stats_collect_period = 10
stream_idle_timeout = "4h0m0s"
stream_server_address = "127.0.0.1"
stream_server_port = "0"
systemd_cgroup = false
tolerate_missing_hugetlb_controller = true
unset_seccomp_profile = ""
[plugins."io.containerd.grpc.v1.cri".cni]
bin_dir = "/opt/cni/bin"
conf_dir = "/etc/cni/net.d"
conf_template = ""
ip_pref = ""
max_conf_num = 1
[plugins."io.containerd.grpc.v1.cri".containerd]
default_runtime_name = "runc"
disable_snapshot_annotations = true
discard_unpacked_layers = false
ignore_rdt_not_enabled_errors = false
no_pivot = false
snapshotter = "overlayfs"
[plugins."io.containerd.grpc.v1.cri".containerd.default_runtime]
base_runtime_spec = ""
cni_conf_dir = ""
cni_max_conf_num = 0
container_annotations = []
pod_annotations = []
privileged_without_host_devices = false
runtime_engine = ""
runtime_path = ""
runtime_root = ""
runtime_type = ""
[plugins."io.containerd.grpc.v1.cri".containerd.default_runtime.options]
[plugins."io.containerd.grpc.v1.cri".containerd.runtimes]
[plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runc]
base_runtime_spec = ""
cni_conf_dir = ""
cni_max_conf_num = 0
container_annotations = []
pod_annotations = []
privileged_without_host_devices = false
runtime_engine = ""
runtime_path = ""
runtime_root = ""
runtime_type = "io.containerd.runc.v2"
[plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runc.options]
BinaryName = ""
CriuImagePath = ""
CriuPath = ""
CriuWorkPath = ""
IoGid = 0
IoUid = 0
NoNewKeyring = false
NoPivotRoot = false
Root = ""
ShimCgroup = ""
SystemdCgroup = false
[plugins."io.containerd.grpc.v1.cri".containerd.untrusted_workload_runtime]
base_runtime_spec = ""
cni_conf_dir = ""
cni_max_conf_num = 0
container_annotations = []
pod_annotations = []
privileged_without_host_devices = false
runtime_engine = ""
runtime_path = ""
runtime_root = ""
runtime_type = ""
[plugins."io.containerd.grpc.v1.cri".containerd.untrusted_workload_runtime.options]
[plugins."io.containerd.grpc.v1.cri".image_decryption]
key_model = "node"
[plugins."io.containerd.grpc.v1.cri".registry]
config_path = ""
[plugins."io.containerd.grpc.v1.cri".registry.auths]
[plugins."io.containerd.grpc.v1.cri".registry.configs]
[plugins."io.containerd.grpc.v1.cri".registry.headers]
[plugins."io.containerd.grpc.v1.cri".registry.mirrors]
[plugins."io.containerd.grpc.v1.cri".x509_key_pair_streaming]
tls_cert_file = ""
tls_key_file = ""
[plugins."io.containerd.internal.v1.opt"]
path = "/opt/containerd"
[plugins."io.containerd.internal.v1.restart"]
interval = "10s"
[plugins."io.containerd.internal.v1.tracing"]
sampling_ratio = 1.0
service_name = "containerd"
[plugins."io.containerd.metadata.v1.bolt"]
content_sharing_policy = "shared"
[plugins."io.containerd.monitor.v1.cgroups"]
no_prometheus = false
[plugins."io.containerd.runtime.v1.linux"]
no_shim = false
runtime = "runc"
runtime_root = ""
shim = "containerd-shim"
shim_debug = false
[plugins."io.containerd.runtime.v2.task"]
platforms = ["linux/amd64"]
sched_core = false
[plugins."io.containerd.service.v1.diff-service"]
default = ["walking"]
[plugins."io.containerd.service.v1.tasks-service"]
rdt_config_file = ""
[plugins."io.containerd.snapshotter.v1.aufs"]
root_path = ""
[plugins."io.containerd.snapshotter.v1.btrfs"]
root_path = ""
[plugins."io.containerd.snapshotter.v1.devmapper"]
async_remove = false
base_image_size = ""
discard_blocks = false
fs_options = ""
fs_type = ""
pool_name = ""
root_path = ""
[plugins."io.containerd.snapshotter.v1.native"]
root_path = ""
[plugins."io.containerd.snapshotter.v1.overlayfs"]
root_path = ""
upperdir_label = false
[plugins."io.containerd.snapshotter.v1.zfs"]
root_path = ""
[plugins."io.containerd.tracing.processor.v1.otlp"]
endpoint = ""
insecure = false
protocol = ""
[proxy_plugins]
[stream_processors]
[stream_processors."io.containerd.ocicrypt.decoder.v1.tar"]
accepts = ["application/vnd.oci.image.layer.v1.tar+encrypted"]
args = ["--decryption-keys-path", "/etc/containerd/ocicrypt/keys"]
env = ["OCICRYPT_KEYPROVIDER_CONFIG=/etc/containerd/ocicrypt/ocicrypt_keyprovider.conf"]
path = "ctd-decoder"
returns = "application/vnd.oci.image.layer.v1.tar"
[stream_processors."io.containerd.ocicrypt.decoder.v1.tar.gzip"]
accepts = ["application/vnd.oci.image.layer.v1.tar+gzip+encrypted"]
args = ["--decryption-keys-path", "/etc/containerd/ocicrypt/keys"]
env = ["OCICRYPT_KEYPROVIDER_CONFIG=/etc/containerd/ocicrypt/ocicrypt_keyprovider.conf"]
path = "ctd-decoder"
returns = "application/vnd.oci.image.layer.v1.tar+gzip"
[timeouts]
"io.containerd.timeout.bolt.open" = "0s"
"io.containerd.timeout.shim.cleanup" = "5s"
"io.containerd.timeout.shim.load" = "5s"
"io.containerd.timeout.shim.shutdown" = "3s"
"io.containerd.timeout.task.state" = "2s"
[ttrpc]
address = ""
gid = 0
uid = 0 vim /etc/containerd/config.tomli进入插入修改，修改如下部分（在plugins."io.containerd.grpc.v1.cri".registry.mirrors部分添加），加入镜像源，修改完成后:wq保存并退出
[plugins."io.containerd.grpc.v1.cri".registry.mirrors]
[plugins."io.containerd.grpc.v1.cri".registry.mirrors."docker.io"]
endpoint = ["https://dockerhub.mirrors.nwafu.edu.cn"]
[plugins."io.containerd.grpc.v1.cri".registry.mirrors."k8s.gcr.io"]
endpoint = ["https://registry.aliyuncs.com/k8sxio"]- registry.mirrors."xxx" : 表示需要配置 mirror 的镜像仓库。例如，registry.mirrors."docker.io" 表示配置 docker.io 的 mirror。
- endpoint : 表示提供 mirror 的镜像加速服务。例如，这里推荐使用西北农林科技大学提供的镜像加速服务作为 docker.io 的 mirror。
启动Containerd
systemctl daemon-reload
systemctl enable containerd
systemctl restart containerd执行完成未出现错误，可直接跳过报错解决和重新安装部分。直接开始安装k8s和Docker
报错解决
执行到这里，我的Containerd出现一个错误，无法正常重启，并且查询报错出现
containerd Job for containerd.service failed because a timeout was exceeded.我查询错误的具体原因是failed to load cni during init, please check CRI plugin status before settin...
我尝试查询解决方案，未能成功解决，欢迎大家提问和讨论，给出解决方案！
错误处理
这里后面在写文章下半部分发现未修改配置文件的k8s.gcr.io会出现错误
执行如下命令修改文件中的地址未阿里云地址
sed -i "s#k8s.gcr.io#registry.cn-hangzhou.aliyuncs.com/google_containers#g" /etc/containerd/config.toml后来只能重新安装，卸载原来的Containerd版本，删除/etc/containerd文件夹
yum remove containerd
rm -rf /etc/containerd重新安装
这次我未用yum安装，使用wget下载安装包安装
wget https://download.fastgit.org/containerd/containerd/releases/download/v1.4.3/cri-containerd-cni-1.4.3-linux-amd64.tar.gz解压，这里下载的1.4.3版本，需要其它版本请修改版本号
sudo tar -C / -xzf cri-containerd-cni-1.4.3-linux-amd64.tar.gz追加到配置文件并使之生效
export PATH=$PATH:/usr/local/bin:/usr/local/sbin
source ~/.bashrc查询Containerd版本
ctr version查询出现一个错误
ctr: failed to dial "/run/containerd/containerd.sock": context deadline exceeded重启Containerd即可
systemctl restart containerd[root@master ~]# ctr version
Client:
Version: v1.4.3
Revision: 269548fa27e0089a8b8278fc4fc781d7f65a939b
Go version: go1.15.5
ctr: failed to dial "/run/containerd/containerd.sock": context deadline exceeded
[root@master ~]# systemctl restart containerd
[root@master ~]# ctr version
Client:
Version: v1.4.3
Revision: 269548fa27e0089a8b8278fc4fc781d7f65a939b
Go version: go1.15.5
Server:
Version: v1.4.3
Revision: 269548fa27e0089a8b8278fc4fc781d7f65a939b
UUID: e4c49cb6-919e-4093-b787-e6835710f1a0继续上述步骤中的配置Containerd
mkdir /etc/containerd
containerd config default > /etc/containerd/config.toml镜像加速的配置就在 cri 插件配置块下面的 registry 配置块
配置文件如下
[root@master ~]# cat /etc/containerd/config.toml
version = 2
root = "/var/lib/containerd"
state = "/run/containerd"
plugin_dir = ""
disabled_plugins = []
required_plugins = []
oom_score = 0
[grpc]
address = "/run/containerd/containerd.sock"
tcp_address = ""
tcp_tls_cert = ""
tcp_tls_key = ""
uid = 0
gid = 0
max_recv_message_size = 16777216
max_send_message_size = 16777216
[ttrpc]
address = ""
uid = 0
gid = 0
[debug]
address = ""
uid = 0
gid = 0
level = ""
[metrics]
address = ""
grpc_histogram = false
[cgroup]
path = ""
[timeouts]
"io.containerd.timeout.shim.cleanup" = "5s"
"io.containerd.timeout.shim.load" = "5s"
"io.containerd.timeout.shim.shutdown" = "3s"
"io.containerd.timeout.task.state" = "2s"
[plugins]
[plugins."io.containerd.gc.v1.scheduler"]
pause_threshold = 0.02
deletion_threshold = 0
mutation_threshold = 100
schedule_delay = "0s"
startup_delay = "100ms"
[plugins."io.containerd.grpc.v1.cri"]
disable_tcp_service = true
stream_server_address = "127.0.0.1"
stream_server_port = "0"
stream_idle_timeout = "4h0m0s"
enable_selinux = false
selinux_category_range = 1024
sandbox_image = "k8s.gcr.io/pause:3.2"
stats_collect_period = 10
systemd_cgroup = false
enable_tls_streaming = false
max_container_log_line_size = 16384
disable_cgroup = false
disable_apparmor = false
restrict_oom_score_adj = false
max_concurrent_downloads = 3
disable_proc_mount = false
unset_seccomp_profile = ""
tolerate_missing_hugetlb_controller = true
disable_hugetlb_controller = true
ignore_image_defined_volumes = false
[plugins."io.containerd.grpc.v1.cri".containerd]
snapshotter = "overlayfs"
default_runtime_name = "runc"
no_pivot = false
disable_snapshot_annotations = true
discard_unpacked_layers = false
[plugins."io.containerd.grpc.v1.cri".containerd.default_runtime]
runtime_type = ""
runtime_engine = ""
runtime_root = ""
privileged_without_host_devices = false
base_runtime_spec = ""
[plugins."io.containerd.grpc.v1.cri".containerd.untrusted_workload_runtime]
runtime_type = ""
runtime_engine = ""
runtime_root = ""
privileged_without_host_devices = false
base_runtime_spec = ""
[plugins."io.containerd.grpc.v1.cri".containerd.runtimes]
[plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runc]
runtime_type = "io.containerd.runc.v2"
runtime_engine = ""
runtime_root = ""
privileged_without_host_devices = false
base_runtime_spec = ""
[plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runc.options]
[plugins."io.containerd.grpc.v1.cri".cni]
bin_dir = "/opt/cni/bin"
conf_dir = "/etc/cni/net.d"
max_conf_num = 1
conf_template = ""
[plugins."io.containerd.grpc.v1.cri".registry]
[plugins."io.containerd.grpc.v1.cri".registry.mirrors]
[plugins."io.containerd.grpc.v1.cri".registry.mirrors."docker.io"]
endpoint = ["https://registry-1.docker.io"]
[plugins."io.containerd.grpc.v1.cri".image_decryption]
key_model = ""
[plugins."io.containerd.grpc.v1.cri".x509_key_pair_streaming]
tls_cert_file = ""
tls_key_file = ""
[plugins."io.containerd.internal.v1.opt"]
path = "/opt/containerd"
[plugins."io.containerd.internal.v1.restart"]
interval = "10s"
[plugins."io.containerd.metadata.v1.bolt"]
content_sharing_policy = "shared"
[plugins."io.containerd.monitor.v1.cgroups"]
no_prometheus = false
[plugins."io.containerd.runtime.v1.linux"]
shim = "containerd-shim"
runtime = "runc"
runtime_root = ""
no_shim = false
shim_debug = false
[plugins."io.containerd.runtime.v2.task"]
platforms = ["linux/amd64"]
[plugins."io.containerd.service.v1.diff-service"]
default = ["walking"]
[plugins."io.containerd.snapshotter.v1.devmapper"]
root_path = ""
pool_name = ""
base_image_size = ""
async_remove = false修改镜像
vim /etc/containerd/config.toml
修改为如下配置文件格式
[plugins."io.containerd.grpc.v1.cri".registry.mirrors]
[plugins."io.containerd.grpc.v1.cri".registry.mirrors."docker.io"]
endpoint = ["https://dockerhub.mirrors.nwafu.edu.cn"]
[plugins."io.containerd.grpc.v1.cri".registry.mirrors."k8s.gcr.io"]
endpoint = ["https://registry.aliyuncs.com/k8sxio"]
保存并退出
启动Containerd，并查询状态
systemctl daemon-reload
systemctl enable containerd
systemctl restart containerd
systemctl status containerd[root@node1 ~]# systemctl daemon-reload
[root@node1 ~]# systemctl enable containerd
Created symlink from /etc/systemd/system/multi-user.target.wants/containerd.service to /etc/systemd/system/containerd.service.
[root@node1 ~]# systemctl restart containerd
[root@node1 ~]# systemctl status containerd
● containerd.service - containerd container runtime
Loaded: loaded (/etc/systemd/system/containerd.service; enabled; vendor preset: disabled)
Active: active (running) since 日 2022-07-31 14:01:53 CST; 8s ago
Docs: https://containerd.io
Process: 61077 ExecStartPre=/sbin/modprobe overlay (code=exited, status=0/SUCCESS)
Main PID: 61080 (containerd)
Tasks: 8
Memory: 56.3M
CGroup: /system.slice/containerd.service
└─61080 /usr/local/bin/containerd
7月 31 14:01:53 node1 containerd[61080]: time="2022-07-31T14:01:53.252033825+08:00" level=info msg="loading plugin \...rpc.v1
7月 31 14:01:53 node1 containerd[61080]: time="2022-07-31T14:01:53.253187854+08:00" level=info msg=serving... addres....ttrpc
7月 31 14:01:53 node1 containerd[61080]: time="2022-07-31T14:01:53.253258189+08:00" level=info msg=serving... addres...d.sock
7月 31 14:01:53 node1 containerd[61080]: time="2022-07-31T14:01:53.260183023+08:00" level=info msg="containerd succe...2919s"
7月 31 14:01:53 node1 containerd[61080]: time="2022-07-31T14:01:53.273749848+08:00" level=info msg="Start subscribin...event"
7月 31 14:01:53 node1 containerd[61080]: time="2022-07-31T14:01:53.273888661+08:00" level=info msg="Start recovering state"
7月 31 14:01:53 node1 containerd[61080]: time="2022-07-31T14:01:53.274076640+08:00" level=info msg="Start event monitor"
7月 31 14:01:53 node1 containerd[61080]: time="2022-07-31T14:01:53.274097447+08:00" level=info msg="Start snapshots syncer"
7月 31 14:01:53 node1 containerd[61080]: time="2022-07-31T14:01:53.274112469+08:00" level=info msg="Start cni networ...yncer"
7月 31 14:01:53 node1 containerd[61080]: time="2022-07-31T14:01:53.274123474+08:00" level=info msg="Start streaming server"
Hint: Some lines were ellipsized, use -l to show in full.安装Containerd完成！
由于文章到这里过长，避免大家的观感疲劳，文章分为上下两个部分，下版部分是安装Dockers+K8s的部分，请看文章
kubeadm部署指定版本的K8s+containerd+docker——图文详细版（下）