k8s常见问题大收集-转载

第1章 k8s外部不能访问pod
1、问题描述：
在搭建好的k8s集群内创建的容器，只能在其所在的节点上curl可访问，但是在其他任何主机上无法访问容器占用的端口
1.1、解决方案

vim /etc/sysctl.conf
找到这一行，放开注释
# Uncomment the next line to enable packet forwarding for IPv4
net.ipv4.ip_forward=1
1
2
3
4
重启主机（必须要重启才能生效）
第2章 创建私有仓库问题
2.1、问题描述，提示需要https协议问题解决

[root@docker docker]# docker push 10.0.0.10:5000/test/nginx:v1
The push refers to repository [10.0.0.10:5000/test/nginx]
Get https://10.0.0.10:5000/v2/: http: server gave HTTP response to HTTPS client
1
2
3
2.1.1、解决方法1：（docker 1.2以上版本解决方法）

在/etc/docker/daemon.json添加以下信息
{ "insecure-registries":["10.0.0.10:5000"]   必须要加在第一行
重启docker，重启registry
systemctl restart docker.service
1
2
3
4
2.1.2、解决方法2：（docker1.2以下版本解决方法）

报错信息2：
[root@lnmp ~]# docker pull 10.0.0.10:5000/test/nginx:v1
Error response from daemon: invalid registry endpoint https://10.0.0.10:5000/v0/: unable to ping registry endpoint https://10.0.0.10:5000/v0/
v2 ping attempt failed with error: Get https://10.0.0.10:5000/v2/: tls: oversized record received with length 20527
 v1 ping attempt failed with error: Get https://10.0.0.10:5000/v1/_ping: tls: oversized record received with length 20527. If this private registry supports only HTTP or HTTPS with an unknown CA certificate, please add `--insecure-registry 10.0.0.10:5000` to the daemon's arguments. In the case of HTTPS, if you have access to the registry's CA certificate, no need for the flag; simply place the CA certificate at /etc/docker/certs.d/10.0.0.10:5000/ca.crt
1
2
3
4
5
2.2、解决办法：

在/etc/sysconfig/docker中添加如下信息即可
other_args="--insecure-registry 10.0.0.10:5000"     私有仓库地址
other_args="--insecure-registry registry:5000"      公有仓库地址
重启docker，重启registry
/etc/init.d/docker restart
1
2
3
4
5
第3章 下载镜像出现问题
3.1、问题1：提示/etc/rhsm/ca/redhat-uep.pem no file or dirctory
3.1.1、解决方法：
3.1.1.1、yum安装需要的依赖包

yum -y install *rhsm*
1
3.1.1.2、下载python-rhsm-certificates软件并生成密钥文件

wget http://mirror.centos.org/centos/7/os/x86_64/Packages/python-rhsm-certificates-1.19.10-1.el7_4.x86_64.rpm
生成密钥
rpm2cpio python-rhsm-certificates-1.19.10-1.el7_4.x86_64.rpm | cpio -iv --to-stdout ./etc/rhsm/ca/redhat-uep.pem | tee /etc/rhsm/ca/redhat-uep.pem
1
2
3
3.1.1.3、重新pull镜像

docker pull registry.access.redhat.com/rhel7/pod-infrastructure:latest
1
第4章 不能删除容器
4.1、docker报错rpc error: code = 14 desc = grpc: the connection is unavailable
4.1.1、尝试关闭容器，进入容器操作界面也报相同错误：

[root@k8s-node-1 ~]# docker exec -it 7119f8f5feef /bin/bash
rpc error: code = 14 desc = grpc: the connection is unavailable
1
2
4.1.1.2、停止容器依旧提示错误

[root@k8s-node-1 ~]# docker stop 7119f8f5feef
Error response from daemon: Cannot stop container 7119f8f5feef: Cannot kill container 7119f8f5feef4c649d9ec04734e6224e2d837fa030de271f269f0b71eea29327: rpc error: code = 14 desc = grpc: the connection is unavailable
1
2
4.1.1.3、删除容器依旧提示错误（-f强制删除）

[root@k8s-node-1 ~]# docker rm -f 7119f8f5feef
Error response from daemon: Could not kill running container 7119f8f5feef4c649d9ec04734e6224e2d837fa030de271f269f0b71eea29327, cannot remove - Cannot kill container 7119f8f5feef4c649d9ec04734e6224e2d837fa030de271f269f0b71eea29327: rpc error: code = 14 desc = grpc: the connection is unavailable
1
2
4.2、解决办法：
4.2.1、使用docker-containerd命令以debug模式调试容器
注意：那个node上的容器不能删除就在那台node上面执行以下命令

[root@k8s-node-1 ~]# docker-containerd -l unix:///var/run/docker/libcontainerd/docker-containerd.sock --metrics-interval=0 --start-timeout 2m --state-dir /var/run/docker/libcontainerd/containerd --shim docker-containerd-shim --runtime docker-runc --debug
WARN[0000] containerd: low RLIMIT_NOFILE changing to max  current=1024 max=4096
DEBU[0000] containerd: read past events                  count=1
 low RLIMIT_NOFILE changing to max  current=1024 max=4096DEBU[0000] containerd: grpc api on /var/run/docker/libcontainerd/docker-containerd.sock 
DEBU[0000] containerd: container restored                id=354af53914e3f76e653a26d9e9da8d4fbef4ef18cc2176371b89871a9126a646
DEBU[0000] containerd: container restored                id=3f0bf43f7ca97c439b64370cee09205b35e58ed35e49f957412f58affbe4ed4b
DEBU[0000] containerd: container restored                id=4b848d33a32a332635929b95eb7291abeb32f177a3c65248568b959dbfbc2712
DEBU[0000] containerd: container restored                id=4ed8d1f971a0ea5035b507511d802a1445af9e771cde670814104102a7cc2d6f
ERRO[0000] containerd: notify OOM events                 error=open /proc/13541/cgroup: no such file or directory
DEBU[0000] containerd: container restored                id=7119f8f5feef4c649d9ec04734e6224e2d837fa030de271f269f0b71eea29327
ERRO[0000] containerd: notify OOM events                 error=open /proc/12860/cgroup: no such file or directory
DEBU[0000] containerd: container restored                id=7bdba0a1ee81997bdbb5958e31123538ac8a6730c6cc7120fe7359439b52b410
DEBU[0000] containerd: container restored                id=8ba79a79836b4350335375f89fc1473a6a86593375fbac6344fb17e4dddff43f
DEBU[0000] containerd: container restored                id=9692f3570460186de681476bd068d008891b24b3906f190443f24e97343c3e57
DEBU[0000] containerd: supervisor running                cpus=1 memory=977 runtime=docker-runc runtimeArgs=[] stateDir=/var/run/docker/libcontainerd/containerd
DEBU[0000] containerd: process exited                    id=7119f8f5feef4c649d9ec04734e6224e2d837fa030de271f269f0b71eea29327 pid=init status=143 systemPid=13541
ERRO[0000] containerd: deleting container                error=exit status 1: "container 7119f8f5feef4c649d9ec04734e6224e2d837fa030de271f269f0b71eea29327 does not exist\none or more of the container deletions failed\n"
DEBU[0000] containerd: process exited                    id=7bdba0a1ee81997bdbb5958e31123538ac8a6730c6cc7120fe7359439b52b410 pid=init status=137 systemPid=12860
ERRO[0000] containerd: deleting container                error=exit status 1: "container 7bdba0a1ee81997bdbb5958e31123538ac8a6730c6cc7120fe7359439b52b410 does not exist\none or more of the container deletions failed\n"

^CINFO[0056] stopping containerd after receiving interrupt
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
4.2.2、调试后发现容器状态变为了未开启，尝试删除容器，成功

docker exec -it 3e22bd0b6a40 /bin/bash
Error response from daemon: Container 3e22bd0b6a40c85d2af45b5d65fb3648acab7e0ad05fa909201051a8f00a3d15 is not running
docker rm -f zen_mclean 
zen_mclean
1
2
3
4
第5章 k8s下DNS问题
5.1、kubelet提示DNS错误信息

kubelet does not have ClusterDNS IP configured and cannot create Pod using "ClusterFirst" policy. Fail
1
5.2、解决办法：

在cat /etc/kubernetes/kubelet 配置文件中添加如下内容即可
KUBE_ARGS="--cluster-dns=10.0.0.110 --cluster-domain=cluster.local"
重启 systemctl daemon-reload;  systemctl restart kubelet 即可
1
2
3
第6章 docker run （镜像）报错，文件系统不支持
1、报错信息如下：

/usr/bin/docker-current: Error response from daemon: error creating overlay mount to /var/lib/docker/overlay2/7b4a1ef8a539785fde3fa4cabc4bb9d90967a30calid argument.
See '/usr/bin/docker-current run --help'.
1
2
2、报错原因

这个是因为用的overlay2文件系统，而系统默认只能识别overlay文件系统
所以我们就要更新文件系统了
1
2
3、解决方法：

systemctl stop docker              //停掉docker服务
rm -rf /var/lib/docker             //注意会清掉docker images的镜像
vi /etc/sysconfig/docker-storage   //将文件里的overlay2改成overlay即可
DOCKER_STORAGE_OPTIONS="--storage-driver overlay2 "  #修改前
DOCKER_STORAGE_OPTIONS="--storage-driver overlay "   #修改后
vi /etc/sysconfig/docker           //去掉option后面的--selinux-enabled
1
2
3
4
5
6
4、重新启动docker即可

systemctl start docker
1
第7章 docker运行apache报错
7.1、报错信息如下：

[root@k8s-node-3 ~]# docker logs 99e3fc059214
WordPress not found in /var/www/html - copying now...
Complete! WordPress has been successfully copied to /var/www/html
AH00534: apache2: Configuration error: No MPM loaded.
1
2
3
4
7.2、解决方法：

systemctl stop docker              //停掉docker服务
rm -rf /var/lib/docker             //注意会清掉docker images的镜像
vi /etc/sysconfig/docker-storage   //将文件里的overlay2改成devicemapper即可
DOCKER_STORAGE_OPTIONS="--storage-driver overlay2 "  #修改前
DOCKER_STORAGE_OPTIONS="--storage-driver devicemapper "   #修改后
1
2
3
4
5
7.3、重启docker服务

systemctl start docker
1
第8章 启动pod报错信息如下：

[root@k8s_master k8s_yaml]# kubectl -n ingress-nginx get events  #通过事件查看错误信息

Warning   FailedCreate        ReplicaSet   Error creating: pods "nginx-ingress-controller-9fc7f4c5f-5f2k4" is forbidden: SecurityContext.RunAsUser is forbidden
7m42s       Warning   FailedCreate        ReplicaSet   Error creating: pods "nginx-ingress-controller-9fc7f4c5f-25wr7" is forbidden: SecurityContext.RunAsUser is forbidden
1
2
3
4
8.1、解决办法：

修改apiserver配置文件，将SecurityContextDeny去掉，重启kube-apiserver即可解决
————————————————
版权声明：本文为CSDN博主「ljx1528」的原创文章，遵循CC 4.0 BY-SA版权协议，转载请附上原文出处链接及本声明。
原文链接：https://blog.csdn.net/ljx1528/article/details/81437106