部署 etcd 3 节点集群

1. 准备csr文件

etcd-csr.json

{
  "CN": "etcd",
  "hosts": [
    "127.0.0.1",
    "192.168.220.121",
    "192.168.220.122",
    "192.168.220.123"
  ],
  "key": {
    "algo": "rsa",
    "size": 2048
  },
  "names": [
    {
      "CN": "CN",
      "ST": "BeiJing",
      "O": "k8s",
      "OU": "yngwie",
      "L": "BeiJing"
    }
  ]
}

2. 准备签名配置文件

ca-config.json

{
  "signing": {
    "default": {
      "expiry": "87600h"
  },
  "profiles": {
    "k8s": {
      "usages": [
          "signing",
          "key encipherment",
          "server auth",
          "client auth"
        ],
        "expiry": "876000h"
      }
    }
  }
}

3.生成etcd证书和私钥(已事先准备好ca证书和私钥)

$ cfssl gencert -ca=ca.crt \
-ca-key=ca.key \
-config=ca-config.json \
-profile=k8s etcd-csr.json | cfssljson -bare etcd

4. 把证书分发到各个节点的/etc/etcd，创建/var/lib/etcd目录

5. 在3个节点启动etcd

./etcd --advertise-client-urls=https://192.168.220.121:2479 --cert-file=etcd.pem --client-cert-auth=true --data-dir=/var/lib/etcd1 --initial-advertise-peer-urls=https://192.168.220.121:2480 --initial-cluster=e1=https://192.168.220.121:2480,e2=https://192.168.220.122:2480,e3=https://192.168.220.123:2480 --key-file=etcd-key.pem --listen-client-urls=https://127.0.0.1:2479,https://192.168.220.121:2479 --listen-metrics-urls=http://127.0.0.1:2481 --listen-peer-urls=https://192.168.220.121:2480 --peer-cert-file=etcd.pem --peer-client-cert-auth=true --peer-key-file=etcd-key.pem --peer-trusted-ca-file=ca.crt --snapshot-count=10000 --trusted-ca-file=ca.crt --name=e1

./etcd --advertise-client-urls=https://192.168.220.122:2479 --cert-file=etcd.pem --client-cert-auth=true --data-dir=/var/lib/etcd --initial-advertise-peer-urls=https://192.168.220.122:2480 --initial-cluster=e1=https://192.168.220.121:2480,e2=https://192.168.220.122:2480,e3=https://192.168.220.123:2480 --key-file=etcd-key.pem --listen-client-urls=https://127.0.0.1:2479,https://192.168.220.122:2479 --listen-metrics-urls=http://127.0.0.1:2481 --listen-peer-urls=https://192.168.220.122:2480 --peer-cert-file=etcd.pem --peer-client-cert-auth=true --peer-key-file=etcd-key.pem --peer-trusted-ca-file=ca.crt --snapshot-count=10000 --trusted-ca-file=ca.crt --name=e2

./etcd --advertise-client-urls=https://192.168.220.123:2479 --cert-file=etcd.pem --client-cert-auth=true --data-dir=/var/lib/etcd --initial-advertise-peer-urls=https://192.168.220.123:2480 --initial-cluster=e1=https://192.168.220.121:2480,e2=https://192.168.220.122:2480,e3=https://192.168.220.123:2480 --key-file=etcd-key.pem --listen-client-urls=https://127.0.0.1:2479,https://192.168.220.123:2479 --listen-metrics-urls=http://127.0.0.1:2481 --listen-peer-urls=https://192.168.220.123:2480 --peer-cert-file=etcd.pem --peer-client-cert-auth=true --peer-key-file=etcd-key.pem --peer-trusted-ca-file=ca.crt --snapshot-count=10000 --trusted-ca-file=ca.crt --name=e3

6. 查看成员状态

$ ./etcdctl --cacert=ca.crt --cert=etcd.pem --key=etcd-key.pem --endpoints=https://192.168.220.121:2479,https://192.168.220.122:2479,https://192.168.220.123:2479 member list

233107b54f466600, started, e2, https://192.168.220.122:2480, https://192.168.220.122:2479, false
68510ebb26e5951d, started, e3, https://192.168.220.123:2480, https://192.168.220.123:2479, false
a8ed76fdaaccdbca, started, e1, https://192.168.220.121:2480, https://192.168.220.121:2479, false

7. 查看endpoint状态

$ ./etcdctl --cacert=ca.crt --cert=etcd.pem --key=etcd-key.pem --endpoints=https://192.168.220.121:2479,https://192.168.220.122:2479,https://192.168.220.123:2479 endpoint status --write-out=table
+------------------------------+------------------+---------+---------+-----------+------------+-----------+------------+--------------------+--------+
|           ENDPOINT           |        ID        | VERSION | DB SIZE | IS LEADER | IS LEARNER | RAFT TERM | RAFT INDEX | RAFT APPLIED INDEX | ERRORS |
+------------------------------+------------------+---------+---------+-----------+------------+-----------+------------+--------------------+--------+
| https://192.168.220.121:2479 | a8ed76fdaaccdbca |   3.4.0 |   20 kB |     false |      false |        47 |         19 |                 19 |        |
| https://192.168.220.122:2479 | 233107b54f466600 |   3.4.0 |   20 kB |      true |      false |        47 |         19 |                 19 |        |
| https://192.168.220.123:2479 | 68510ebb26e5951d |   3.4.0 |   29 kB |     false |      false |        47 |         19 |                 19 |        |
+------------------------------+------------------+---------+---------+-----------+------------+-----------+------------+--------------------+--------+

8. 备份etcd， 只能备份一个结点

$ ./etcdctl --cacert=ca.crt --cert=etcd.pem --key=etcd-key.pem --endpoints=https://192.168.220.122:2479 snapshot save snapshot-etcd
{"level":"info","ts":1591772525.710396,"caller":"snapshot/v3_snapshot.go:109","msg":"created temporary db file","path":"snapshot-etcd.part"}
{"level":"warn","ts":"2020-06-10T15:02:05.716+0800","caller":"clientv3/retry_interceptor.go:116","msg":"retry stream intercept"}
{"level":"info","ts":1591772525.7161868,"caller":"snapshot/v3_snapshot.go:120","msg":"fetching snapshot","endpoint":"https://192.168.220.122:2479"}
{"level":"info","ts":1591772525.7202048,"caller":"snapshot/v3_snapshot.go:133","msg":"fetched snapshot","endpoint":"https://192.168.220.122:2479","took":0.009444471}
{"level":"info","ts":1591772525.7204468,"caller":"snapshot/v3_snapshot.go:142","msg":"saved","path":"snapshot-etcd"}
Snapshot saved at snapshot-etcd

9. 查看备份

$ ./etcdctl --cacert=ca.crt --cert=etcd.pem --key=etcd-key.pem --endpoints=https://192.168.220.123:2479 snapshot status snapshot-etcd --write-out=table
+----------+----------+------------+------------+
|   HASH   | REVISION | TOTAL KEYS | TOTAL SIZE |
+----------+----------+------------+------------+
| b3c3a9a5 |        2 |          7 |      20 kB |
+----------+----------+------------+------------+