【运维】K8S集群部署系列之ETCD集群搭建（一）
ETCD 普通集群搭建
将从本文开始记录K8S集群部署的全部操作，打造傻瓜式部署教程。
文章目录
- ETCD 普通集群搭建
- 简介
- 系统环境
- 软件环境
- etcd安装
- 下载安装包
- 集群节点配置文件
- master节点配置
- node1节点配置
- node2节点配置
- ETCD服务启动文件
- 网络准备
- 启动集群
- 集群健康检查
简介
ETCD是基于GO语言编写的用于配置共享和服务发现的键值存储系统，以简单、安全、快速、可信为核心。
系统环境
| Node | IP | CentOS | kernel | cpu | memory |
|---|---|---|---|---|---|
| master | 192.168.159.3 | CentOS Linux release 7.4.1708 (Core) | 3.10.0-693.el7.x86_64 | Intel® Core™ i5-7500 CPU @ 3.40GHz * 1 | 2G |
| node1 | 192.168.159.4 | CentOS Linux release 7.4.1708 (Core) | 3.10.0-693.el7.x86_64 | Intel® Core™ i5-7500 CPU @ 3.40GHz * 1 | 2G |
| node2 | 192.168.159.5 | CentOS Linux release 7.4.1708 (Core) | 3.10.0-693.el7.x86_64 | Intel® Core™ i5-7500 CPU @ 3.40GHz * 1 | 2G |
| node3 | 192.168.159.6 | CentOS Linux release 7.4.1708 (Core) | 3.10.0-693.el7.x86_64 | Intel® Core™ i5-7500 CPU @ 3.40GHz * 1 | 2G |
软件环境
| Node | IP | etcd |
|---|---|---|
| master | 192.168.159.3 | 3.3.13 |
| node1 | 192.168.159.4 | 3.3.13 |
| node2 | 192.168.159.5 | 3.3.13 |
etcd安装
ETCD官方文档:https://etcd.io/docs/v3.3.12/op-guide/configuration/
下载安装包
-
下载地址
https://github.com/etcd-io/etcd/releases
-
下载安装包
在四个节点执行以下操作
mkdir /home/k8s cd /home/k8s wget https://github.com/etcd-io/etcd/releases/download/v3.3.13/etcd-v3.3.13-linux-amd64.tar.gz tar -zxvf etcd-v3.3.13-linux-amd64.tar.gz mv etcd-v3.3.13-linux-amd64 etcd chmod -R +x etcd/ cp -f ./{etcd,etcdctl} /usr/bin/ cp -f ./{etcd,etcdctl} /usr/local/bin/
集群节点配置文件
master节点配置
mkdir -p /opt/etcd/{data,etc}
cat > /opt/etcd/etc/etcd.conf << EOF
#[Member]
ETCD_NAME="etcd-1"
ETCD_DATA_DIR="/opt/etcd/data"
ETCD_LISTEN_PEER_URLS="http://192.168.159.3:2380"
ETCD_LISTEN_CLIENT_URLS="http://192.168.159.3:2379,http://127.0.0.1:2379"
#[Clustering]
ETCD_INITIAL_ADVERTISE_PEER_URLS="http://192.168.159.3:2380"
ETCD_ADVERTISE_CLIENT_URLS="http://192.168.159.3:2379"
ETCD_INITIAL_CLUSTER="etcd-1=http://192.168.159.3:2380,etcd-2=http://192.168.159.4:2380,etcd-3=http://192.168.159.5:2380"
ETCD_INITIAL_CLUSTER_TOKEN="etcd-cluster"
ETCD_INITIAL_CLUSTER_STATE="new"
EOF
- 名词解释
ETCD_NAME： 节点名称 ETCD_DATA_DIR: 节点数据目录 ETCD_LISTEN_PEER_URLS： 节点服务端通信地址 ETCD_LISTEN_CLIENT_URLS： 节点客户端通信地址 ETCD_INITIAL_ADVERTISE_PEER_URLS： 节点集群内服务端通信地址 ETCD_ADVERTISE_CLIENT_URLS： 节点集群内客户端通信地址 ETCD_INITIAL_CLUSTER： 集群节点集 ETCD_INITIAL_CLUSTER_TOKEN： token，标识同一集群 ETCD_INITIAL_CLUSTER_STATE： new为新建集群，existing为加入已创建集群，此处为new
node1节点配置
mkdir -p /opt/etcd/{data,etc}
cat > /opt/etcd/etc/etcd.conf << EOF
#[Member]
ETCD_NAME="etcd-2"
ETCD_DATA_DIR="/opt/etcd/data"
ETCD_LISTEN_PEER_URLS="http://192.168.159.4:2380"
ETCD_LISTEN_CLIENT_URLS="http://192.168.159.4:2379,http://127.0.0.1:2379"
#[Clustering]
ETCD_INITIAL_ADVERTISE_PEER_URLS="http://192.168.159.4:2380"
ETCD_ADVERTISE_CLIENT_URLS="http://192.168.159.4:2379"
ETCD_INITIAL_CLUSTER="etcd-1=http://192.168.159.3:2380,etcd-2=http://192.168.159.4:2380,etcd-3=http://192.168.159.5:2380"
ETCD_INITIAL_CLUSTER_TOKEN="etcd-cluster"
ETCD_INITIAL_CLUSTER_STATE="new"
EOF
node2节点配置
mkdir -p /opt/etcd/{data,etc}
cat > /opt/etcd/etc/etcd.conf << EOF
#[Member]
ETCD_NAME="etcd-3"
ETCD_DATA_DIR="/opt/etcd/data"
ETCD_LISTEN_PEER_URLS="http://192.168.159.5:2380"
ETCD_LISTEN_CLIENT_URLS="http://192.168.159.5:2379,http://127.0.0.1:2379"
#[Clustering]
ETCD_INITIAL_ADVERTISE_PEER_URLS="http://192.168.159.5:2380"
ETCD_ADVERTISE_CLIENT_URLS="http://192.168.159.5:2379"
ETCD_INITIAL_CLUSTER="etcd-1=http://192.168.159.3:2380,etcd-2=http://192.168.159.4:2380,etcd-3=http://192.168.159.5:2380"
ETCD_INITIAL_CLUSTER_TOKEN="etcd-cluster"
ETCD_INITIAL_CLUSTER_STATE="new"
EOF
ETCD服务启动文件
systemd中文手册:http://www.jinbuguo.com/systemd/systemd.exec.html
cat > vim /usr/lib/systemd/system/etcd.service << EOF
[Unit]
Description=Etcd Server
After=network.target
After=network-online.target
Wants=network-online.target
[Service]
Type=notify
EnvironmentFile=-/opt/etcd/etc/etcd.conf
ExecStart=/home/k8s/etcd/etcd \
--name=${ETCD_NAME} \
--data-dir=${ETCD_DATA_DIR} \
--listen-peer-urls=${ETCD_LISTEN_PEER_URLS} \
--listen-client-urls=${ETCD_LISTEN_CLIENT_URLS} \
--initial-advertise-peer-urls=${ETCD_INITIAL_ADVERTISE_PEER_URLS} \
--advertise-client-urls=${ETCD_ADVERTISE_CLIENT_URLS} \
--initial-cluster=${ETCD_INITIAL_CLUSTER} \
--initial-cluster-token=${ETCD_INITIAL_CLUSTER_TOKEN} \
--initial-cluster-state=${ETCD_INITIAL_CLUSTER_STATE}
Restart=on-failure
LimitNOFILE=65536
[Install]
WantedBy=multi-user.target
EOF
网络准备
cat >>/etc/hosts<< EOF
192.168.159.3 master
192.168.159.4 node1
192.168.159.5 node2
192.168.159.6 node3
EOF
# 防火墙设置，开放2379和2380端口,如果启动防火墙但未放开端口则集群状态为“degraded”，
# 开启防火墙的节点状态为“are all unreachable”
# 2379端口提供给客户端访问集群，客户端如：etcdctl
# 2380端口提供给集群节点间通信
systemctl start firewalld
firewall-cmd --zone=public --add-port=2379/tcp --permanent
firewall-cmd --zone=public --add-port=2380/tcp --permanent
firewall-cmd --reload
firewall-cmd --list-all
启动集群
systemctl start etcd
注意: 集群节点需要同时启动才能成功，否则会报错某个节点无法找到；如果是重新创建集群，则需要删除旧的数据目录。
集群健康检查
etcdctl cluster-health
member 8ada33a16cb8b5f9 is healthy: got healthy result from http://192.168.159.4:2379
member df5c33b8666738a6 is healthy: got healthy result from http://192.168.159.3:2379
member e689a191b9fab04f is healthy: got healthy result from http://192.168.159.5:2379
cluster is healthy # cluster is degraded表示集群至少有一个节点不可达
etcdctl member list
8ada33a16cb8b5f9: name=etcd-2 peerURLs=http://192.168.159.4:2380 clientURLs=http://192.168.159.4:2379 isLeader=true # 主节点
df5c33b8666738a6: name=etcd-1 peerURLs=http://192.168.159.3:2380 clientURLs=http://192.168.159.3:2379 isLeader=false
e689a191b9fab04f: name=etcd-3 peerURLs=http://192.168.159.5:2380 clientURLs=http://192.168.159.5:2379 isLeader=false
至此
ETCD数据库普通集群搭建完毕；
下一篇将介绍TLS证书的制作。