k8s版本:1.26.0
flannel:0.20.2
flannel-cni-plugin:v1.1.0
flannel是k8s常用的网络插件,正常的部署步骤为:
https://github.com/flannel-io/flannel
kubectl apply -f https://github.com/flannel-io/flannel/releases/latest/download/kube-flannel.yml
但是在国内如果按照该步骤,则会显示
[root@k8s-node1 ~]# kubectl get pod -n kube-flannel
NAMESPACE NAME READY STATUS RESTARTS AGE
kube-flannel kube-flannel-ds-2wz5z 0/1 Init:ImagePullBackOff 0 13s
kube-flannel kube-flannel-ds-n6wcb 0/1 Init:ImagePullBackOff 0 13s
kube-flannel kube-flannel-ds-wk62c 0/1 Init:ImagePullBackOff 0 13s
此时查看任意pod报错原因,显示为:
[root@k8s-node1 ~]# kubectl describe pod kube-flannel-ds-2wz5z -n kube-flannel
Name: kube-flannel-ds-2wz5z
Namespace: kube-flannel
Priority: 2000001000
Priority Class Name: system-node-critical
Service Account: flannel
Node: k8s-node3/10.15.0.23
Start Time: Thu, 22 Jun 2023 13:57:58 +0800
Labels: app=flannel
controller-revision-hash=6d89ffc7b6
pod-template-generation=1
tier=node
Annotations: <none>
Status: Pending
IP: 10.15.0.23
IPs:
IP: 10.15.0.23
Controlled By: DaemonSet/kube-flannel-ds
Init Containers:
install-cni-plugin:
Container ID:
Image: docker.io/rancher/mirrored-flannelcni-flannel-cni-plugin:v1.1.0
Image ID:
Port: <none>
Host Port: <none>
Command:
cp
Args:
-f
/flannel
/opt/cni/bin/flannel
State: Waiting
Reason: ImagePullBackOff
Ready: False
Restart Count: 0
Environment: <none>
Mounts:
/opt/cni/bin from cni-plugin (rw)
/var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-nj6jn (ro)
install-cni:
Container ID:
Image: docker.io/rancher/mirrored-flannelcni-flannel:v0.20.2
Image ID:
Port: <none>
Host Port: <none>
Command:
cp
Args:
-f
/etc/kube-flannel/cni-conf.json
/etc/cni/net.d/10-flannel.conflist
State: Waiting
Reason: PodInitializing
Ready: False
Restart Count: 0
Environment: <none>
Mounts:
/etc/cni/net.d from cni (rw)
/etc/kube-flannel/ from flannel-cfg (rw)
/var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-nj6jn (ro)
Containers:
kube-flannel:
Container ID:
Image: docker.io/rancher/mirrored-flannelcni-flannel:v0.20.2
Image ID:
Port: <none>
Host Port: <none>
Command:
/opt/bin/flanneld
Args:
--ip-masq
--kube-subnet-mgr
State: Waiting
Reason: PodInitializing
Ready: False
Restart Count: 0
Limits:
cpu: 100m
memory: 50Mi
Requests:
cpu: 100m
memory: 50Mi
Environment:
POD_NAME: kube-flannel-ds-2wz5z (v1:metadata.name)
POD_NAMESPACE: kube-flannel (v1:metadata.namespace)
EVENT_QUEUE_DEPTH: 5000
Mounts:
/etc/kube-flannel/ from flannel-cfg (rw)
/run/flannel from run (rw)
/run/xtables.lock from xtables-lock (rw)
/var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-nj6jn (ro)
Conditions:
Type Status
Initialized False
Ready False
ContainersReady False
PodScheduled True
Volumes:
run:
Type: HostPath (bare host directory volume)
Path: /run/flannel
HostPathType:
cni-plugin:
Type: HostPath (bare host directory volume)
Path: /opt/cni/bin
HostPathType:
cni:
Type: HostPath (bare host directory volume)
Path: /etc/cni/net.d
HostPathType:
flannel-cfg:
Type: ConfigMap (a volume populated by a ConfigMap)
Name: kube-flannel-cfg
Optional: false
xtables-lock:
Type: HostPath (bare host directory volume)
Path: /run/xtables.lock
HostPathType: FileOrCreate
kube-api-access-nj6jn:
Type: Projected (a volume that contains injected data from multiple sources)
TokenExpirationSeconds: 3607
ConfigMapName: kube-root-ca.crt
ConfigMapOptional: <nil>
DownwardAPI: true
QoS Class: Burstable
Node-Selectors: <none>
Tolerations: :NoSchedule op=Exists
node.kubernetes.io/disk-pressure:NoSchedule op=Exists
node.kubernetes.io/memory-pressure:NoSchedule op=Exists
node.kubernetes.io/network-unavailable:NoSchedule op=Exists
node.kubernetes.io/not-ready:NoExecute op=Exists
node.kubernetes.io/pid-pressure:NoSchedule op=Exists
node.kubernetes.io/unreachable:NoExecute op=Exists
node.kubernetes.io/unschedulable:NoSchedule op=Exists
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Scheduled 49s default-scheduler Successfully assigned kube-flannel/kube-flannel-ds-2wz5z to k8s-node3
Warning FailedMount 48s kubelet MountVolume.SetUp failed for volume "kube-api-access-nj6jn" : failed to sync configmap cache: timed out waiting for the condition
Normal Pulling 47s kubelet Pulling image "docker.io/rancher/mirrored-flannelcni-flannel-cni-plugin:v1.1.0"
Warning Failed 22s kubelet Failed to pull image "docker.io/rancher/mirrored-flannelcni-flannel-cni-plugin:v1.1.0": rpc error: code = Unknown desc = failed to pull and unpack image "docker.io/rancher/mirrored-flannelcni-flannel-cni-plugin:v1.1.0": failed to resolve reference "docker.io/rancher/mirrored-flannelcni-flannel-cni-plugin:v1.1.0": failed to do request: Head "https://registry-1.docker.io/v2/rancher/mirrored-flannelcni-flannel-cni-plugin/manifests/v1.1.0": net/http: TLS handshake timeout
Warning Failed 22s kubelet Error: ErrImagePull
Normal SandboxChanged 15s (x7 over 21s) kubelet Pod sandbox changed, it will be killed and re-created.
Normal BackOff 15s (x7 over 21s) kubelet Back-off pulling image "docker.io/rancher/mirrored-flannelcni-flannel-cni-plugin:v1.1.0"
Warning Failed 15s (x7 over 21s) kubelet Error: ImagePullBackOff
原因也很明显,k8s在拉取docker.io/rancher/mirrored-flannelcni-flannel-cni-plugin:v1.1.0
时失败了,默认拉取的镜像仓库在国外,被墙了
作为一个完全不懂k8s和docker的新手,我琢磨出以下曲线解决方案
这一步网上资料很多,不再赘述,只要保证执行docker info
后Registry Mirrors
中能看到国内镜像源,执行docker pull hello-world
成功即可
[root@k8s-node1 ~]# docker info
Client: Docker Engine - Community
Version: 24.0.2
Context: default
Debug Mode: false
Plugins:
buildx: Docker Buildx (Docker Inc.)
Version: v0.10.5
Path: /usr/libexec/docker/cli-plugins/docker-buildx
compose: Docker Compose (Docker Inc.)
Version: v2.18.1
Path: /usr/libexec/docker/cli-plugins/docker-compose
Server:
Containers: 0
Running: 0
Paused: 0
Stopped: 0
Images: 3
Server Version: 24.0.2
Storage Driver: overlay2
Backing Filesystem: xfs
Supports d_type: true
Using metacopy: false
Native Overlay Diff: true
userxattr: false
Logging Driver: json-file
Cgroup Driver: cgroupfs
Cgroup Version: 1
Plugins:
Volume: local
Network: bridge host ipvlan macvlan null overlay
Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
Swarm: inactive
Runtimes: io.containerd.runc.v2 runc
Default Runtime: runc
Init Binary: docker-init
containerd version: 3dce8eb055cbb6872793272b4f20ed16117344f8
runc version: v1.1.7-0-g860f061
init version: de40ad0
Security Options:
seccomp
Profile: builtin
Kernel Version: 3.10.0-1062.el7.x86_64
Operating System: CentOS Linux 7 (Core)
OSType: linux
Architecture: x86_64
CPUs: 2
Total Memory: 1.777GiB
Name: k8s-node1
ID: 30778c8d-86b5-4c16-84e9-990ea7cf4d22
Docker Root Dir: /var/lib/docker
Debug Mode: false
Experimental: false
Insecure Registries:
127.0.0.0/8
Registry Mirrors:
https://dockerproxy.com/
https://hub-mirror.c.163.com/
https://mirror.baidubce.com/
https://ccr.ccs.tencentyun.com/
Live Restore Enabled: false
[root@k8s-node1 containerd]# docker pull hello-world
Using default tag: latest
latest: Pulling from library/hello-world
Digest: sha256:c2e23624975516c7e27b1b25be3682a8c6c4c0cea011b791ce98aa423b5040a0
Status: Image is up to date for hello-world:latest
docker.io/library/hello-world:latest
[root@k8s-node1 containerd]# docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
hello-world latest 9c7a54a9a43c 6 weeks ago 13.3kB
在步骤2中,我们展示了命令kubectl apply -f https://github.com/flannel-io/flannel/releases/latest/download/kube-flannel.yml
,实际上是kubectl
在线拉取了https://github.com/flannel-io/flannel/releases/latest/download/kube-flannel.yml
,我们可以wget或者直接网页打开该地址,获取该yaml文件内容,保存到本地,文件名仍然为kube-flannel.yml
,此时执行grep image kube-flannel.yml
可得到需要的镜像的地址,此时就可以用docker pull
进行下载了
[root@k8s-node1 ~]# grep image kube-flannel.yml
#image: flannelcni/flannel-cni-plugin:v1.1.0 for ppc64le and mips64le (dockerhub limitations may apply)
image: docker.io/rancher/mirrored-flannelcni-flannel-cni-plugin:v1.1.0
#image: flannelcni/flannel:v0.20.2 for ppc64le and mips64le (dockerhub limitations may apply)
image: docker.io/rancher/mirrored-flannelcni-flannel:v0.20.2
#image: flannelcni/flannel:v0.20.2 for ppc64le and mips64le (dockerhub limitations may apply)
image: docker.io/rancher/mirrored-flannelcni-flannel:v0.20.2
[root@k8s-node1 ~]# docker pull docker.io/rancher/mirrored-flannelcni-flannel-cni-plugin:v1.1.0
v1.1.0: Pulling from rancher/mirrored-flannelcni-flannel-cni-plugin
Digest: sha256:28d3a6be9f450282bf42e4dad143d41da23e3d91f66f19c01ee7fd21fd17cb2b
Status: Image is up to date for rancher/mirrored-flannelcni-flannel-cni-plugin:v1.1.0
docker.io/rancher/mirrored-flannelcni-flannel-cni-plugin:v1.1.0
[root@k8s-node1 ~]# docker pull docker.io/rancher/mirrored-flannelcni-flannel:v0.20.2
v0.20.2: Pulling from rancher/mirrored-flannelcni-flannel
Digest: sha256:ec0f0b7430c8370c9f33fe76eb0392c1ad2ddf4ccaf2b9f43995cca6c94d3832
Status: Image is up to date for rancher/mirrored-flannelcni-flannel:v0.20.2
docker.io/rancher/mirrored-flannelcni-flannel:v0.20.2
这里使用的阿里云个人镜像仓库,网上有很多教程如何注册,不再赘述
此处使用的广州
节点,然后随便脸滚键盘写个命名空间ahiusydqiu
,然后在里面建立仓库flannel
和flannel-cni-plugin
,再将步骤3.2下载的镜像上传到仓库中
[root@k8s-node1 containerd]# docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
hello-world latest 9c7a54a9a43c 6 weeks ago 13.3kB
rancher/mirrored-flannelcni-flannel v0.20.2 b5c6c9203f83 6 months ago 59.6MB
rancher/mirrored-flannelcni-flannel-cni-plugin v1.1.0 fcecffc7ad4a 13 months ago 8.09MB
[root@k8s-node1 containerd]# docker tag rancher/mirrored-flannelcni-flannel:v0.20.2 registry.cn-guangzhou.aliyuncs.com/ahiusydqiu/flannel:v0.20.2
[root@k8s-node1 containerd]# docker push registry.cn-guangzhou.aliyuncs.com/ahiusydqiu/flannel:v0.20.2
[root@k8s-node1 containerd]# docker tag rancher/mirrored-flannelcni-flannel-cni-plugin:v1.1.0 registry.cn-guangzhou.aliyuncs.com/ahiusydqiu/flannel-cni-plugin:v1.1.0
[root@k8s-node1 containerd]# docker push registry.cn-guangzhou.aliyuncs.com/ahiusydqiu/flannel-cni-plugin:v1.1.0
对比步骤3.2,更改后的kube-flannel.yml
的image为:
[root@k8s-node1 ~]# grep image kube-flannel.yml
#image: flannelcni/flannel-cni-plugin:v1.1.0 for ppc64le and mips64le (dockerhub limitations may apply)
#image: docker.io/rancher/mirrored-flannelcni-flannel-cni-plugin:v1.1.0
image: registry.cn-guangzhou.aliyuncs.com/ahiusydqiu/flannel-cni-plugin:v1.1.0
#image: flannelcni/flannel:v0.20.2 for ppc64le and mips64le (dockerhub limitations may apply)
#image: docker.io/rancher/mirrored-flannelcni-flannel:v0.20.2
image: registry.cn-guangzhou.aliyuncs.com/ahiusydqiu/flannel:v0.20.2
#image: flannelcni/flannel:v0.20.2 for ppc64le and mips64le (dockerhub limitations may apply)
#image: docker.io/rancher/mirrored-flannelcni-flannel:v0.20.2
image: registry.cn-guangzhou.aliyuncs.com/ahiusydqiu/flannel:v0.20.2
[root@k8s-node1 ~]# kubectl apply -f kube-flannel.yml
namespace/kube-flannel created
clusterrole.rbac.authorization.k8s.io/flannel created
clusterrolebinding.rbac.authorization.k8s.io/flannel created
serviceaccount/flannel created
configmap/kube-flannel-cfg created
daemonset.apps/kube-flannel-ds created
[root@k8s-node1 ~]# kubectl get pod -n kube-flannel
NAMESPACE NAME READY STATUS RESTARTS AGE
kube-flannel kube-flannel-ds-h7pvv 1/1 Running 0 21m
kube-flannel kube-flannel-ds-q5d5x 1/1 Running 0 21m
kube-flannel kube-flannel-ds-tzfns 1/1 Running 0 21m
如果拉取不下来,使用kubectl describe pod kube-flannel-ds-mp2j6 -n kube-flannel
中报错信息有denied等字样,可能是阿里云仓库中设为私有仓库,设为公有仓库即可