Kubernetes部署（四）：ETCD集群部署

相关内容：

Kubernetes部署（一）：架构及功能说明
Kubernetes部署（二）：系统环境初始化
Kubernetes部署（三）：CA证书制作
Kubernetes部署（四）：ETCD集群部署
Kubernetes部署（五）：Haproxy、Keppalived部署
Kubernetes部署（六）：Master节点部署
Kubernetes部署（七）：Node节点部署
Kubernetes部署（八）：Flannel网络部署
Kubernetes部署（九）：CoreDNS、Dashboard、Ingress部署
Kubernetes部署（十）：储存之glusterfs和heketi部署
Kubernetes部署（十一）：管理之Helm和Rancher部署
Kubernetes部署（十二）：helm部署harbor企业级镜像仓库

手动部署ETCD集群

配置前3台主机需时间同步。

0.准备etcd软件包

[root@node-01 k8s]# wget https://github.com/coreos/etcd/releases/download/v3.2.18/etcd-v3.2.18-linux-amd64.tar.gz
[root@node-01 k8s]# tar zxf etcd-v3.2.18-linux-amd64.tar.gz
[root@node-01 k8s]# cd etcd-v3.2.18-linux-amd64
[root@node-01 etcd-v3.2.18-linux-amd64]# cp etcd etcdctl /data/kubernetes/bin/
#拷贝到其他2个master节点
[root@node-01 etcd-v3.2.18-linux-amd64]# scp etcd etcdctl 10.31.90.202:/data/kubernetes/bin/
[root@node-01 etcd-v3.2.18-linux-amd64]# scp etcd etcdctl 10.31.90.203:/data/kubernetes/bin/

1.创建 etcd 证书签名请求：

这里需要列出所有需要安装etcd节点的ip，不然会导致后续报错。

[root@node-01 ssl]# vim etcd-csr.json
{
  "CN": "etcd",
  "hosts": [
    "127.0.0.1",
    "10.31.90.201",
    "10.31.90.202",
    "10.31.90.203"
  ],
  "key": {
    "algo": "rsa",
    "size": 2048
  },
  "names": [
    {
      "C": "CN",
      "ST": "BeiJing",
      "L": "BeiJing",
      "O": "k8s",
      "OU": "System"
    }
  ]
}

2.生成 etcd 证书和私钥：

[root@node-01 ~]# cfssl gencert -ca=/data/kubernetes/ssl/ca.pem \
  -ca-key=/data/kubernetes/ssl/ca-key.pem \
  -config=/data/kubernetes/ssl/ca-config.json \
  -profile=kubernetes etcd-csr.json | cfssljson -bare etcd
会生成以下证书文件
[root@node-01 ssl]# ls -l etcd*
-rw-r--r-- 1 root root 1062 Dec 24 17:33 etcd.csr
-rw-r--r-- 1 root root  296 Dec 24 17:32 etcd-csr.json
-rw------- 1 root root 1679 Dec 24 17:33 etcd-key.pem
-rw-r--r-- 1 root root 1436 Dec 24 17:33 etcd.pem

3.将证书移动到master节点的/data/kubernetes/ssl目录下

[root@node-01 ssl]# cp etcd*.pem /data/kubernetes/ssl/
[root@node-01 ssl]# scp etcd*.pem 10.31.90.202:/data/kubernetes/ssl/
[root@node-01 ssl]# scp etcd*.pem 10.31.90.203:/data/kubernetes/ssl/
[root@node-01 ssl]# rm -f etcd.csr etcd-csr.json 

4.设置ETCD配置文件

[root@node-01 k8s]# vim /data/kubernetes/cfg/etcd.conf
#[member]
ETCD_NAME="etcd-node1"
ETCD_DATA_DIR="/var/lib/etcd/default.etcd"
#ETCD_SNAPSHOT_COUNTER="10000"
#ETCD_HEARTBEAT_INTERVAL="100"
#ETCD_ELECTION_TIMEOUT="1000"
ETCD_LISTEN_PEER_URLS="https://10.31.90.201:2380"
ETCD_LISTEN_CLIENT_URLS="https://10.31.90.201:2379,https://127.0.0.1:2379"
#ETCD_MAX_SNAPSHOTS="5"
#ETCD_MAX_WALS="5"
#ETCD_CORS=""
#[cluster]
ETCD_INITIAL_ADVERTISE_PEER_URLS="https://10.31.90.201:2380"
# if you use different ETCD_NAME (e.g. test),
# set ETCD_INITIAL_CLUSTER value for this name, i.e. "test=http://..."
ETCD_INITIAL_CLUSTER="etcd-node1=https://10.31.90.201:2380,etcd-node2=https://10.31.90.202:2380,etcd-node3=https://10.31.90.203:2380"
ETCD_INITIAL_CLUSTER_STATE="new"
ETCD_INITIAL_CLUSTER_TOKEN="k8s-etcd-cluster"
ETCD_ADVERTISE_CLIENT_URLS="https://10.31.90.201:2379"
#[security]
CLIENT_CERT_AUTH="true"
ETCD_CA_FILE="/data/kubernetes/ssl/ca.pem"
ETCD_CERT_FILE="/data/kubernetes/ssl/etcd.pem"
ETCD_KEY_FILE="/data/kubernetes/ssl/etcd-key.pem"
PEER_CLIENT_CERT_AUTH="true"
ETCD_PEER_CA_FILE="/data/kubernetes/ssl/ca.pem"
ETCD_PEER_CERT_FILE="/data/kubernetes/ssl/etcd.pem"
ETCD_PEER_KEY_FILE="/data/kubernetes/ssl/etcd-key.pem"

5.创建ETCD系统服务

[root@node-01 k8s]# vim /etc/systemd/system/etcd.service
[Unit]
Description=Etcd Server
After=network.target

[Service]
Type=simple
WorkingDirectory=/var/lib/etcd
EnvironmentFile=-/data/kubernetes/cfg/etcd.conf
# set GOMAXPROCS to number of processors
ExecStart=/bin/bash -c "GOMAXPROCS=$(nproc) /data/kubernetes/bin/etcd"
Type=notify

[Install]
WantedBy=multi-user.target

6.重新加载系统服务

[root@node-01 k8s]# systemctl daemon-reload
[root@node-01 k8s]# systemctl enable etcd

[root@node-01 k8s]# scp /data/kubernetes/cfg/etcd.conf 10.31.90.202:/data/kubernetes/cfg/
[root@node-01 k8s]# scp /data/kubernetes/cfg/etcd.conf 10.31.90.203:/data/kubernetes/cfg/
[root@node-01 k8s]# scp /etc/systemd/system/etcd.service 10.31.90.202:/etc/systemd/system/
[root@node-01 k8s]# scp /etc/systemd/system/etcd.service 10.31.90.203:/etc/systemd/system/
在所有节点上创建etcd存储目录并启动etcd
[root@node-01 ~]# mkdir /var/lib/etcd
[root@node-01 ~]# systemctl start etcd
[root@node-01 ~]# systemctl status etcd

下面需要大家在所有的 etcd 节点重复上面的步骤，并修改对应的配置文件，直到所有机器的 etcd 服务都已启动。

7.验证集群

[root@node-01 k8s]# etcdctl --endpoints=https://10.31.90.201:2379 \
>   --ca-file=/data/kubernetes/ssl/ca.pem \
>   --cert-file=/data/kubernetes/ssl/etcd.pem \
>   --key-file=/data/kubernetes/ssl/etcd-key.pem cluster-health
member 6d88194636f8d570 is healthy: got healthy result from https://10.31.90.202:2379
member ad9448fdb3b3cd26 is healthy: got healthy result from https://10.31.90.203:2379
member fb7fb38e5e81969b is healthy: got healthy result from https://10.31.90.201:2379
cluster is healthy

后续会陆续更新所有的安装文档，如果你觉得我写的不错，希望大家多多关注点赞，非常感谢！