K8s-部署 etcd 集群.04

内容转载自:https://github.com/opsnull/follow-me-install-kubernetes-cluster/blob/master/04.%E9%83%A8%E7%BD%B2etcd%E9%9B%86%E7%BE%A4.md

上一篇:K8s-部署 kubectl 命令行工具.03

04.部署 etcd 集群

etcd 是基于 Raft 的分布式 key-value 存储系统,由 CoreOS 开发,常用于服务发现、共享配置以及并发控制(如 leader 选举、分布式锁等)。kubernetes 使用 etcd 存储所有运行数据。

本文档介绍部署一个三节点高可用 etcd 集群的步骤:

  • 下载和分发 etcd 二进制文件;
  • 创建 etcd 集群各节点的 x509 证书,用于加密客户端(如 etcdctl) 与 etcd 集群、etcd 集群之间的数据流;
  • 创建 etcd 的 systemd unit 文件,配置服务参数;
  • 检查集群工作状态;

etcd 集群各节点的名称和 IP 如下:

  • k8s-etcd-0001:172.27.129.104
  • k8s-etcd-0002:172.27.129.105
  • k8s-etcd-0003:172.27.129.106

下载和分发 etcd 二进制文件

到 https://github.com/coreos/etcd/releases 页面下载最新版本的发布包:

wget https://github.com/coreos/etcd/releases/download/v3.3.7/etcd-v3.3.7-linux-amd64.tar.gz
tar -xvf etcd-v3.3.7-linux-amd64.tar.gz

分发二进制文件到集群etcd节点:

source /opt/k8s/bin/environment.sh
for etcd_ip in ${ETCD_IP[@]}
  do
    echo ">>> ${netcd_ip}"
    scp etcd-v3.3.7-linux-amd64/etcd* k8s@${etcd_ip}:/opt/k8s/bin
    ssh k8s@${etcd_ip} "chmod +x /opt/k8s/bin/*"
  done

创建 etcd 证书和私钥

cat > etcd-csr.json <
  • hosts 字段指定授权使用该证书的 etcd 节点 IP 或域名列表,这里将 etcd 集群的三个节点 IP 都列在其中;

生成证书和私钥:

cfssl gencert -ca=/etc/kubernetes/cert/ca.pem \
    -ca-key=/etc/kubernetes/cert/ca-key.pem \
    -config=/etc/kubernetes/cert/ca-config.json \
    -profile=kubernetes etcd-csr.json | cfssljson -bare etcd
ls etcd*

分发生成的证书和私钥到各 etcd 节点:

source /opt/k8s/bin/environment.sh
for etcd_ip in ${ETCD_IP[@]}
  do
    echo ">>> ${etcd_ip}"
    ssh root@${etcd_ip} "mkdir -p /etc/etcd/cert && chown -R k8s /etc/etcd/cert"
    scp etcd*.pem k8s@${etcd_ip}:/etc/etcd/cert/
  done

创建 etcd 的 systemd unit 模板文件

source /opt/k8s/bin/environment.sh
cat > etcd.service.template <
  • User:指定以 k8s 账户运行;
  • WorkingDirectory--data-dir:指定工作目录和数据目录为 /var/lib/etcd,需在启动服务前创建这个目录;
  • --name:指定节点名称,当 --initial-cluster-state 值为 new 时,--name 的参数值必须位于 --initial-cluster 列表中;
  • --cert-file--key-file:etcd server 与 client 通信时使用的证书和私钥;
  • --trusted-ca-file:签名 client 证书的 CA 证书,用于验证 client 证书;
  • --peer-cert-file--peer-key-file:etcd 与 peer 通信使用的证书和私钥;
  • --peer-trusted-ca-file:签名 peer 证书的 CA 证书,用于验证 peer 证书;

为各节点创建和分发 etcd systemd unit 文件

替换模板文件中的变量,为各节点创建 systemd unit 文件:

source /opt/k8s/bin/environment.sh
for (( i=0; i < 3; i++ ))
  do
    sed -e "s/##ETCD_NAME##/${ETCD_NAMES[i]}/" -e "s/##ETCD_IP##/${ETCD_IP[i]}/" etcd.service.template > etcd-${ETCD_IP[i]}.service 
  done
ls *.service
  • ETCD_NAMES 和 ETCD_IP 为相同长度的 bash 数组,分别为节点名称和对应的 IP;

分发生成的 systemd unit 文件:

source /opt/k8s/bin/environment.sh
for etcd_ip in ${ETCD_IP[@]}
  do
    echo ">>> ${etcd_ip}"
    ssh root@${etcd_ip} "mkdir -p /var/lib/etcd && chown -R k8s /var/lib/etcd" 
    scp etcd-${etcd_ip}.service root@${etcd_ip}:/etc/systemd/system/etcd.service
  done
  • 必须先创建 etcd 数据目录和工作目录;
  • 文件重命名为 etcd.service;

启动 etcd 服务

source /opt/k8s/bin/environment.sh
for etcd_ip in ${ETCD_IP[@]}
  do
    echo ">>> ${etcd_ip}"
    ssh root@${etcd_ip} "systemctl daemon-reload && systemctl enable etcd && systemctl restart etcd"
  done
  • etcd 进程首次启动时会等待其它节点的 etcd 加入集群,命令 systemctl restart etcd 会卡住一段时间,为正常现象。
  • (如卡住,可Ctrl+c中断命令重新执行一次循环)

检查启动结果

source /opt/k8s/bin/environment.sh
for etcd_ip in ${ETCD_IP[@]}
  do
    echo ">>> ${etcd_ip}"
    ssh k8s@${etcd_ip} "systemctl status etcd|grep Active"
  done

确保状态为 active (running),否则查看日志,确认原因:

$ journalctl -u etcd

验证服务状态

注意:部署完 etcd 集群后,在任一 etcd(k8s-etcd-0001到k8s-etcd-0003) 节点上执行如下命令:

source /opt/k8s/bin/environment.sh
for etcd_ip in ${ETCD_IP[@]}
  do
    echo ">>> ${etcd_ip}"
    ETCDCTL_API=3 /opt/k8s/bin/etcdctl \
    --endpoints=https://${etcd_ip}:2379 \
    --cacert=/etc/kubernetes/cert/ca.pem \
    --cert=/etc/etcd/cert/etcd.pem \
    --key=/etc/etcd/cert/etcd-key.pem endpoint health
  done

预期输出:

https://172.27.129.104:2379 is healthy: successfully committed proposal: took = 2.192932ms
https://172.27.129.105:2379 is healthy: successfully committed proposal: took = 3.546896ms
https://172.27.129.106:2379 is healthy: successfully committed proposal: took = 3.013667ms

输出均为 healthy 时表示集群服务正常。

下一篇:K8s-部署flannel网络.05

你可能感兴趣的:(K8s-部署 etcd 集群.04)