k8s集群部署中etcd启动报错request sent was ignored (cluster ID mismatch: peer[c39bdec535db1fd5]=cdf818194e3a8c
k8s集群部署中etcd启动报错处理
- 报错信息如下，主要报错信息有两条
- 原因是/var/lib/etcd/目录下缓存导致
- 解决方法
- 问题2： Error starting daemon: SELinux is not supported with the overlay2 graph driver on this kernel. Either boot into a newer kernel or disable selinux
-
- 报错如下
报错信息如下，主要报错信息有两条
1、request sent was ignored (cluster ID mismatch: peer[c39bdec535db1fd5]=cdf818194e3a8c32, local=b0daaba520989844)
2、Error: client: etcd cluster is unavailable or misconfigured; error #0: dial tcp 127.0.0.1:4001: connect: connection refused
; error #1: client: endpoint http://127.0.0.1:2379 exceeded header timeout
详细的报错信息见下面两段代码
[root@localhost etcd]# systemctl restart etcd.service
Job for etcd.service failed because a timeout was exceeded. See "systemctl status etcd.service" and "journalctl -xe" for details.
[root@localhost etcd]# journalctl -xe
Jun 18 01:09:29 localhost.localdomain etcd[62325]: request sent was ignored (cluster ID mismatch: peer[c39bdec535db1fd5]=cdf818194e3a8c32, local=b0daaba520989844)
Jun 18 01:09:29 localhost.localdomain etcd[62325]: request sent was ignored (cluster ID mismatch: peer[c39bdec535db1fd5]=cdf818194e3a8c32, local=b0daaba520989844)
Jun 18 01:09:29 localhost.localdomain etcd[62325]: request sent was ignored (cluster ID mismatch: peer[c39bdec535db1fd5]=cdf818194e3a8c32, local=b0daaba520989844)
Jun 18 01:09:29 localhost.localdomain etcd[62325]: request sent was ignored (cluster ID mismatch: peer[c39bdec535db1fd5]=cdf818194e3a8c32, local=b0daaba520989844)
Jun 18 01:09:29 localhost.localdomain etcd[62325]: request sent was ignored (cluster ID mismatch: peer[c39bdec535db1fd5]=cdf818194e3a8c32, local=b0daaba520989844)
Jun 18 01:09:29 localhost.localdomain etcd[62325]: request sent was ignored (cluster ID mismatch: peer[c39bdec535db1fd5]=cdf818194e3a8c32, local=b0daaba520989844)
Jun 18 01:09:29 localhost.localdomain etcd[62325]: request sent was ignored (cluster ID mismatch: peer[c39bdec535db1fd5]=cdf818194e3a8c32, local=b0daaba520989844)
Jun 18 01:09:29 localhost.localdomain etcd[62325]: request sent was ignored (cluster ID mismatch: peer[c39bdec535db1fd5]=cdf818194e3a8c32, local=b0daaba520989844)
Jun 18 01:09:29 localhost.localdomain etcd[62325]: request sent was ignored (cluster ID mismatch: peer[c39bdec535db1fd5]=cdf818194e3a8c32, local=b0daaba520989844)
Jun 18 01:09:29 localhost.localdomain etcd[62325]: request sent was ignored (cluster ID mismatch: peer[c39bdec535db1fd5]=cdf818194e3a8c32, local=b0daaba520989844)
Jun 18 01:09:29 localhost.localdomain etcd[62325]: request sent was ignored (cluster ID mismatch: peer[c39bdec535db1fd5]=cdf818194e3a8c32, local=b0daaba520989844)
Jun 18 01:09:29 localhost.localdomain etcd[62325]: request sent was ignored (cluster ID mismatch: peer[c39bdec535db1fd5]=cdf818194e3a8c32, local=b0daaba520989844)
Jun 18 01:09:29 localhost.localdomain etcd[62325]: request sent was ignored (cluster ID mismatch: peer[c39bdec535db1fd5]=cdf818194e3a8c32, local=b0daaba520989844)
Jun 18 01:09:29 localhost.localdomain etcd[62325]: request sent was ignored (cluster ID mismatch: peer[c39bdec535db1fd5]=cdf818194e3a8c32, local=b0daaba520989844)
Jun 18 01:09:29 localhost.localdomain etcd[62325]: request sent was ignored (cluster ID mismatch: peer[c39bdec535db1fd5]=cdf818194e3a8c32, local=b0daaba520989844)
Jun 18 01:09:29 localhost.localdomain etcd[62325]: request sent was ignored (cluster ID mismatch: peer[c39bdec535db1fd5]=cdf818194e3a8c32, local=b0daaba520989844)
Jun 18 01:09:29 localhost.localdomain etcd[62325]: request sent was ignored (cluster ID mismatch: peer[c39bdec535db1fd5]=cdf818194e3a8c32, local=b0daaba520989844)
Jun 18 01:09:29 localhost.localdomain etcd[62325]: c3342da4612d81d7 is starting a new election at term 133
Jun 18 01:09:29 localhost.localdomain etcd[62325]: c3342da4612d81d7 became candidate at term 134
Jun 18 01:09:29 localhost.localdomain etcd[62325]: c3342da4612d81d7 received MsgVoteResp from c3342da4612d81d7 at term 134
Jun 18 01:09:29 localhost.localdomain etcd[62325]: c3342da4612d81d7 [logterm: 1, index: 2] sent MsgVote request to c39bdec535db1fd5 at term 134
[root@localhost ~]# etcdctl cluster-health
cluster may be unhealthy: failed to list members
Error: client: etcd cluster is unavailable or misconfigured; error #0: client: endpoint http://127.0.0.1:2379 exceeded header timeout
; error #1: dial tcp 127.0.0.1:4001: connect: connection refused
error #0: client: endpoint http://127.0.0.1:2379 exceeded header timeout
error #1: dial tcp 127.0.0.1:4001: connect: connection refused
[root@localhost ~]# etcdctl cluster-health
cluster may be unhealthy: failed to list members
Error: client: etcd cluster is unavailable or misconfigured; error #0: dial tcp 127.0.0.1:4001: connect: connection refused
; error #1: client: endpoint http://127.0.0.1:2379 exceeded header timeout
error #0: dial tcp 127.0.0.1:4001: connect: connection refused
error #1: client: endpoint http://127.0.0.1:2379 exceeded header timeout
原因是/var/lib/etcd/目录下缓存导致
找了一堆处理办法，此博客有提到处理办法如下
https://my.oschina.net/u/4274516/blog/3522268
【坑1】
场景1：报错:rafthttp: request sent was ignored (cluster ID mismatch: peer[b6568aca930d28d4]=cdf818194e3a8c32, local=9c8b920197d88342)
场景2：当某个peer挂了，或者减少etcd集群的成员等，你变更了配置之后，发现重启失败等等之类的错误，都可以尝试如下的解决办法
解决：
cd /var/lib/etcd/
rm ./* -rf //清缓存
解决方法
但是没有说清楚，需要**把master和node节点的/var/lib/etcd/目录下的缓存都删除一遍，然后重启etcd**
已经亲测有效。
问题2： Error starting daemon: SELinux is not supported with the overlay2 graph driver on this kernel. Either boot into a newer kernel or disable selinux
报错如下
Jun 18 02:00:41 localhost.localdomain dockerd-current[75401]: Error starting daemon: SELinux is not supported with the overlay2 graph driver on this kernel. Either boot into a newer kernel or disable selinux i
Jun 18 02:00:41 localhost.localdomain systemd[1]: docker.service: main process exited, code=exited, status=1/FAILURE
Jun 18 02:00:41 localhost.localdomain systemd[1]: Failed to start Docker Application Container Engine.
##处理办法
1、升级内核
yum update
2、没有启动新的内核，修改的docker配置文件。将配置文件的“–selinux-enabled”改成“–selinux-enabled=false”，然后再重启docker。
[root@registry lib]# cat /etc/sysconfig/docker
# /etc/sysconfig/docker
# Modify these options if you want to change the way the docker daemon runs
#OPTIONS='--selinux-enabled --log-driver=journald --signature-verification=false'
OPTIONS='--selinux-enabled=false --log-driver=journald --signature-verification=false --registry-mirror=https://fzhifedh.mirror.aliyuncs.com --insecure-registry=registry.sese.com' #修改这里的"--selinux-enabled"，改成"--selinux-enabled=false"
if [ -z "${DOCKER_CERT_PATH}" ]; then
DOCKER_CERT_PATH=/etc/docker
fi
...... #配置文件后面的内容省略
[root@registry lib]#