给 kubernetes 集群配置外部 etcd 集群(kubeadm)

官方文档： https://kubernetes.io/docs/setup/independent/high-availability/#first-steps-for-both-methods

一、环境准备

ip	hostname	role
10.127.24.180	k8s-node1	k8s-node
10.127.24.179	k8s-master1	k8s-master
10.39.5.226	k8s-node2	k8s-node
10.39.14.204	etcd-node1	etcd-node1
10.39.14.205	etcd-node2	etcd-node2
10.39.14.206	etcd-node3	etcd-node3

开始前保证：

• 所有节点都可以 ping 通
• 都有 sudo 权限
• 所有节点安装了 kubeadm kubelet。kubectl可选。
• etcd 集群已搭建
• 所有节点都有 etcd

二、master 启动集群

1、将 etcd 的认证文件 copy 来

copy 认证文件方法：
1、 先把之前的集群信息都清除干净
2、 跳板机把 etcd 的认证文件 copy 下来
3、 把认证文件放到指定位置

[跳板机]$ scp -r ~/host1 root@:~
[跳板机]$ ssh root@
kubeadm reset
rm -rf /etc/kubernetes/pki/
mkdir -p  /etc/kubernetes/pki/etcd/
chown -R root:root ~/host1/pki/
mv ~/host1/pki/etcd/ca.crt /etc/kubernetes/pki/etcd/
mv ~/host1/pki/apiserver-etcd-client.crt /etc/kubernetes/pki/
mv ~/host1/pki/apiserver-etcd-client.key /etc/kubernetes/pki/
rm -rf ~/host1/

2、创建一个 kubeadm-config.yaml：

参数参考 https://godoc.org/k8s.io/kubernetes/cmd/kubeadm/app/apis/kubeadm/v1beta1

cat < /etc/kubernetes/kubeadm-config.yaml
apiVersion: kubeadm.k8s.io/v1beta1
kind: ClusterConfiguration
kubernetesVersion: v1.13.0
apiServer:
  certSANs:
  - 10.127.24.179
  - 127.0.0.1
networking:
  podSubnet: 10.244.0.0/16
etcd:
    external:
        endpoints:
        - https://10.39.14.204:2379
        - https://10.39.14.205:2379
        - https://10.39.14.206:2379
        caFile: /etc/kubernetes/pki/etcd/ca.crt
        certFile: /etc/kubernetes/pki/apiserver-etcd-client.crt
        keyFile: /etc/kubernetes/pki/apiserver-etcd-client.key
EOF

3、启动集群

kubeadm init --config /etc/kubernetes/kubeadm-config.yaml

记录：
kubeadm join xxxxx --token xxxxx --discovery-token-ca-cert-hash sha256:xxxx

4、配置 flannel 网络

kubectl apply -f kube-flannel.yml

输出：
clusterrole.rbac.authorization.k8s.io/flannel created
clusterrolebinding.rbac.authorization.k8s.io/flannel created
serviceaccount/flannel created
configmap/kube-flannel-cfg created
daemonset.extensions/kube-flannel-ds-amd64 created
daemonset.extensions/kube-flannel-ds-arm64 created
daemonset.extensions/kube-flannel-ds-arm created
daemonset.extensions/kube-flannel-ds-ppc64le created
daemonset.extensions/kube-flannel-ds-s390x created

5、查看当前集群状态

$ kubectl get componentstatus
NAME                 STATUS    MESSAGE              ERROR
controller-manager   Healthy   ok                   
scheduler            Healthy   ok                   
etcd-1               Healthy   {"health": "true"}   
etcd-2               Healthy   {"health": "true"}   
etcd-0               Healthy   {"health": "true"}
$ kubectl get nodes
NAME                                                    STATUS     ROLES    AGE   VERSION
k8s-master1   Ready   master   66m   v1.13.0
$ kubectl get pods -n kube-system
NAME                                                                            READY   STATUS    RESTARTS   AGE
coredns-86c58d9df4-j8t2t                                                        1/1     Running   0          6m20s
coredns-86c58d9df4-wr9sk                                                        1/1     Running   0          6m20s
etcd-k8s-master1                      1/1     Running   0          5m32s
kube-apiserver-k8s-master1            1/1     Running   0          5m43s
kube-controller-manager-k8s-master1   1/1     Running   0          5m21s
kube-flannel-ds-amd64-st4sv                                                     1/1     Running   0          4m
kube-proxy-d7t4d                                                                1/1     Running   0          6m20s
kube-scheduler-k8s-master1            1/1     Running   0          5m39s

三、Node 加入集群

kubeadm join 10.127.24.179:6443 --token xxxxx --discovery-token-ca-cert-hash sha256:xxxxxx

输出：
This node has joined the cluster!

四、（可选）master ：check 一下 Node 状态

kubectl get nodes

输出：
NAME STATUS ROLES AGE VERSION
k8s-master1 Ready master 3h11m v1.13.0
k8s-node1 Ready 2m29s v1.13.0
k8s-node2 Ready 12s v1.13.0