简介
etcd是一种高可用的、分布式数据库,以键值对的方式存储数据。在kubernetes环境里,用户在master所做的一切比如创建pod、删除pod等操作都记录在etcd里,所安装的flannel或者calico网络的配置也保存在etcd数据库里。
主机清单
角色 | 主机 |
---|---|
etcd1 | 172.16.210.53 |
etcd2 | 172.16.210.54 |
etcd3 | 172.16.210.55 |
部署etcd
三台主机提前写入hosts,关闭防火墙,selinux
1.所有节点安装etcd
yum install -y etcd
2.编辑etcd1主机的配置文件
[root@etcd1 ~]# cd /etc/etcd/
[root@etcd1 etcd]# vim etcd.conf
ETCD_DATA_DIR="/var/lib/etcd/cluster.etcd"
ETCD_LISTEN_PEER_URLS="http://172.16.210.53:2380"
ETCD_LISTEN_CLIENT_URLS="http://172.16.210.53:2379,http://localhost:2379"
ETCD_NAME="etcd1"
ETCD_INITIAL_ADVERTISE_PEER_URLS="http://172.16.210.53:2380"
ETCD_ADVERTISE_CLIENT_URLS="http://172.16.210.53:2379,http://localhost:2379"
ETCD_INITIAL_CLUSTER="etcd1=http://172.16.210.53:2380,etcd2=http://172.16.210.54:2380,etcd3=http://172.16.210.55:2380"
ETCD_INITIAL_CLUSTER_TOKEN="etcd-cluster"
ETCD_INITIAL_CLUSTER_STATE="new"
参数详解
ETCD_NAME 节点名称,默认为default
ETCD_DATA_DIR 服务运行数据保存的路径
ETCD_LISTEN_PEER_URLS 监听的同伴通信的地址,比如http://ip:2380,如果有多个,使用逗号分隔。需
要所有节点都能够访问,所以不要使用 localhost!
ETCD_LISTEN_CLIENT_URLS 监听的客户端服务地址
ETCD_ADVERTISE_CLIENT_URLS 对外公告的该节点客户端监听地址,这个值会告诉集群中其他节点。
ETCD_INITIAL_ADVERTISE_PEER_URLS 对外公告的该节点同伴监听地址,这个值会告诉集群中其他节点
ETCD_INITIAL_CLUSTER 集群中所有节点的信息,格式为
ETCD_INITIAL_CLUSTER_STATE 新建集群的时候,这个值为 new;假如加入已经存在的集群,这个值为
existing。
ETCD_INITIAL_CLUSTER_TOKEN 集群的ID,多个集群的时候,每个集群的ID必须保持唯一
3.拷贝配置文件到其他两个节点
[root@etcd1 etcd]# scp etcd.conf 172.16.210.54:/etc/etcd/
[root@etcd1 etcd]# scp etcd.conf 172.16.210.55:/etc/etcd/
4.修改其他两个节点的配置文件
[root@etcd2 etcd]# vim etcd.conf ####修改etcd节点名称和监听的地址,其他参数保持不变
ETCD_LISTEN_PEER_URLS="http://172.16.210.54:2380"
ETCD_LISTEN_CLIENT_URLS="http://172.16.210.54:2379,http://localhost:2379"
ETCD_NAME="etcd2"
ETCD_INITIAL_ADVERTISE_PEER_URLS="http://172.16.210.54:2380"
ETCD_ADVERTISE_CLIENT_URLS="http://172.16.210.54:2379,http://localhost:2379"
[root@etcd3 etcd]# vim etcd.conf
ETCD_LISTEN_PEER_URLS="http://172.16.210.55:2380"
ETCD_LISTEN_CLIENT_URLS="http://172.16.210.55:2379,http://localhost:2379"
ETCD_NAME="etcd3"
ETCD_INITIAL_ADVERTISE_PEER_URLS="http://172.16.210.55:2380"
ETCD_ADVERTISE_CLIENT_URLS="http://172.16.210.55:2379,http://localhost:2379"
5.重启3个节点的etcd
systemctl restart etcd
6.查看节点状态
[root@etcd1 etcd]# etcdctl member list ##可以看到3个节点都已经加入进来
1e986f07f84e3d22: name=etcd3 peerURLs=http://172.16.210.55:2380 clientURLs=http://172.16.210.55:2379,http://localhost:2379 isLeader=true
78ce7c84d3112610: name=etcd1 peerURLs=http://172.16.210.53:2380 clientURLs=http://172.16.210.53:2379,http://localhost:2379 isLeader=false
91c3d42e32b4b5fe: name=etcd2 peerURLs=http://172.16.210.54:2380 clientURLs=http://172.16.210.54:2379,http://localhost:2379 isLeader=false
7.测试同步性
在节点1创建目录
[root@etcd1 ~]# etcdctl mkdir /aaa
在其他节点查看
[root@etcd2 ~]# etcdctl ls /
/aaa
[root@etcd3 ~]# etcdctl ls /
/aaa
可以看到在etcd1上创的目录可以在其他节点查看到,注意这个/aaa
目录不是系统的目录,是etcd数据库的目录
8.增加一个节点
8.1 4台节点同步hosts映射
[root@etcd1 ~]# cat /etc/hosts
127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4
::1 localhost localhost.localdomain localhost6 localhost6.localdomain6
172.16.210.53 etcd1
172.16.210.54 etcd2
172.16.210.55 etcd3
172.16.210.56 etcd4
8.2 在第4个节点安装etcd
[root@etcd4 ~]# yum install -y etcd
8.3 拷贝配置文件到第4个节点
[root@etcd1 ~]#scp /etc/etcd/etcd.conf 172.16.210.56:/etc/etcd/
8.4 修改配置文件
[root@etcd4 ~]# vim /etc/etcd/etcd.conf
TCD_DATA_DIR="/var/lib/etcd/cluster.etcd"
ETCD_LISTEN_PEER_URLS="http://172.16.210.56:2380"
ETCD_LISTEN_CLIENT_URLS="http://172.16.210.56:2379,http://localhost:2379"
ETCD_NAME="etcd4"
ETCD_INITIAL_ADVERTISE_PEER_URLS="http://172.16.210.56:2380"
ETCD_ADVERTISE_CLIENT_URLS="http://172.16.210.56:2379,http://localhost:2379"
ETCD_INITIAL_CLUSTER="etcd1=http://172.16.210.53:2380,etcd2=http://172.16.210.54:2380,etcd3=http://172.16.210.55:2380,etcd4=http://172.16.210.56:2380"
ETCD_INITIAL_CLUSTER_TOKEN="etcd-cluster"
ETCD_INITIAL_CLUSTER_STATE="existing"
8.5 重启etcd
[root@etcd4 ~]# systemctl restart etcd ##有报错可以先不用管
Job for etcd.service failed because the control process exited with error code. See "systemctl status etcd.service" and "journalctl -xe" for details.
8.6在节点1把节点4添加进来
[root@etcd1 ~]# etcdctl member add etcd4 http://172.16.210.56:2380
8.7回到节点4再次重启etcd
[root@etcd4 ~]# systemctl restart etcd
8.8再次查看节点状态
[root@etcd1 ~]# etcdctl member list ##可以看到etcd4节点已经加入进来
235ef0a0ac7f415: name=etcd4 peerURLs=http://172.16.210.56:2380 clientURLs=http://172.16.210.56:2379,http://localhost:2379 isLeader=false
1e986f07f84e3d22: name=etcd3 peerURLs=http://172.16.210.55:2380 clientURLs=http://172.16.210.55:2379,http://localhost:2379 isLeader=true
78ce7c84d3112610: name=etcd1 peerURLs=http://172.16.210.53:2380 clientURLs=http://172.16.210.53:2379,http://localhost:2379 isLeader=false
91c3d42e32b4b5fe: name=etcd2 peerURLs=http://172.16.210.54:2380 clientURLs=http://172.16.210.54:2379,http://localhost:2379 isLeader=false
9.删除节点
命令格式 etcdctl member remove 节点id
[root@etcd1 ~]# etcdctl member remove 235ef0a0ac7f415
Removed member 235ef0a0ac7f415 from cluster
[root@etcd1 ~]# etcdctl member list ##可以看到节点4就已经被删除了
1e986f07f84e3d22: name=etcd3 peerURLs=http://172.16.210.55:2380 clientURLs=http://172.16.210.55:2379,http://localhost:2379 isLeader=true
78ce7c84d3112610: name=etcd1 peerURLs=http://172.16.210.53:2380 clientURLs=http://172.16.210.53:2379,http://localhost:2379 isLeader=false
91c3d42e32b4b5fe: name=etcd2 peerURLs=http://172.16.210.54:2380 clientURLs=http://172.16.210.54:2379,http://localhost:2379 isLeader=false
如果还需要再把节点添加进来.必须要要删除节点的数据目录
[root@etcd4 ~]# rm -rf /var/lib/etcd/cluster.etcd/
然后在节点1把节点4添加进来
[root@etcd1 ~]# etcdctl member add etcd4 http://172.16.210.56:2380
重启节点4的etcd并查看节点状态
[root@etcd4 ~]# systemctl restart etcd
[root@etcd4 ~]# etcdctl member list ##可以看到节点4已经重新加入进来
1e986f07f84e3d22: name=etcd3 peerURLs=http://172.16.210.55:2380 clientURLs=http://172.16.210.55:2379,http://localhost:2379 isLeader=true
630cc3da6c62df70: name=etcd4 peerURLs=http://172.16.210.56:2380 clientURLs=http://172.16.210.56:2379,http://localhost:2379 isLeader=false
78ce7c84d3112610: name=etcd1 peerURLs=http://172.16.210.53:2380 clientURLs=http://172.16.210.53:2379,http://localhost:2379 isLeader=false
91c3d42e32b4b5fe: name=etcd2 peerURLs=http://172.16.210.54:2380 clientURLs=http://172.16.210.54:2379,http://localhost:2379 isLeader=false