为CentOS 7.2中的Kuberbetes集群搭建Dashboard

此前，根据在CentOS 7.2上部署Kubernetes集群 已部署一个可用Kubernetes集群，在这篇文章中将介绍如何为该集群搭建可视化界面。

部署Kubernetes Dashboard

根据kubernetes github的介绍下载部署Kubernetes Dashboard的资源文件：

# wget https://git.io/kube-dashboard-no-rbac
# mv kube-dashboard-no-rbac kube-dashboard-no-rbac.yaml

然后参照下面的内容编辑该文件：

...
        args:
          # Uncomment the following line to manually specify Kubernetes API server Host
          # If not specified, Dashboard will attempt to auto discover the API server and connect
          # to it. Uncomment only if the default does not work.
          - --apiserver-host=http://192.168.120.121:8080
...

最后执行kubectl命令创建pod：

# kubectl create -f kube-dashboard-no-rbac.yaml 
deployment "kubernetes-dashboard" created
service "kubernetes-dashboard" created

根据该资源文件中spec.template.spec.containers.image的值gcr.io/google_containers/kubernetes-dashboard-amd64:v1.6.1 可知用于创建容器的镜像位于境外的仓库，因此必须先配置工作节点可以访问境外的网站。

问题解决

现在，使用kubectl命令检查已创建的pod状态是否正常：

# kubectl get pods --all-namespaces -o wide | grep dashboard
kube-system   kubernetes-dashboard-3951142596-drt8r   0/1       CrashLoopBackOff   14         1m       172.30.103.4   kube-node3

可以看到kubernetes-dashboard-3951142596-drt8r的状态为CrashLoopBackOff，即该pod未正常运行。那么是什么原因导致发生这种情况？在该pod的绑定工作节点kube-node3上查看日志文件/var/log/messages：

...
May 27 10:34:45 kube-node3 journal: Creating in-cluster Heapster client
May 27 10:34:45 kube-node3 journal: E0527 02:34:45.767392       1 config.go:322] Expected to load root CA config from /var/run/secrets/kubernetes.io/serviceaccount/ca.crt, but got err: open /var/run/secrets/kubernetes.io/serviceaccount/ca.crt: no such file or directory
May 27 10:34:45 kube-node3 journal: E0527 02:34:45.767392       1 config.go:322] Expected to load root CA config from /var/run/secrets/kubernetes.io/serviceaccount/ca.crt, but got err: open /var/run/secrets/kubernetes.io/serviceaccount/ca.crt: no such file or directory
...

产生这个错误是因为Kubernetes默认创建的secrets资源不包含用于访问kube-apiserver的根证书。

# kubectl get secrets --namespace=kube-system
NAME                  TYPE                                  DATA      AGE
default-token-wxzm7   kubernetes.io/service-account-token   2         19h

# kubectl describe secret default-token-wxzm7 --namespace=kube-system
Name:       default-token-wxzm7
Namespace:  kube-system
Labels:     
Annotations:    kubernetes.io/service-account.name=default
        kubernetes.io/service-account.uid=80bc5d75-41e9-11e7-b90e-000c29f6f813

Type:   kubernetes.io/service-account-token

Data
====
namespace:  11 bytes
token:      eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJrdWJlcm5ldGVzL3NlcnZpY2VhY2NvdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9uYW1lc3BhY2UiOiJrdWJlLXN5c3RlbSIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VjcmV0Lm5hbWUiOiJkZWZhdWx0LXRva2VuLXd4em03Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9zZXJ2aWNlLWFjY291bnQubmFtZSI6ImRlZmF1bHQiLCJrdWJlcm5ldGVzLmlvL3NlcnZpY2VhY2NvdW50L3NlcnZpY2UtYWNjb3VudC51aWQiOiI4MGJjNWQ3NS00MWU5LTExZTctYjkwZS0wMDBjMjlmNmY4MTMiLCJzdWIiOiJzeXN0ZW06c2VydmljZWFjY291bnQ6a3ViZS1zeXN0ZW06ZGVmYXVsdCJ9.m1dIyZiU2vejuUrhjUb3mwykBkp_nrfTQ9kyz6kYQghcJT4iuGNqh3sPBpQ6F4QxCDu_PgKGWr5A7PA3mnvwfmwE8MbLktizf4khOR7gMxp_xwQw8izutdjQZJgtejxzSkBeW3Kh-Xr7YnUt6cpAdkITWJ65rTI5Fp4KmrK-AVMnKr0h3YIbmCTC2-rKJSJw_NUHLYjCELh8c5K2gnn1wTl6QXhgsojtx7cDZZrPBPF6pOX5xtZYN2YEOjjeHS01LA1jbmkaCJiaTT1umICVpGZ8PxRbuuzaUBAdJaxxsE05Jve67E9e6qFIYROsZMIgnoN5t5UBooypBuMkms_31g

生成证书和密钥

在此，使用easyrsa生产证书和密钥。

• 下载easyrsa3

# curl -L -O https://storage.googleapis.com/kubernetes-release/easy-rsa/easy-rsa.tar.gz
# tar xzf easy-rsa.tar.gz
# cd easy-rsa-master/easyrsa3
# ./easyrsa init-pki

init-pki complete; you may now create a CA or requests.
Your newly created PKI dir is: /root/k8s/easy-rsa-master/easyrsa3/pki

• 创建根证书

# ./easyrsa --batch "--req-cn=192.168.120.121@`date +%s`" build-ca nopass
Generating a 2048 bit RSA private key
.......+++
................................................................................+++
writing new private key to '/root/k8s/easy-rsa-master/easyrsa3/pki/private/ca.key'
-----

• 创建服务端证书和密钥

# ./easyrsa --subject-alt-name="IP:192.168.120.121" build-server-full server nopass
Generating a 2048 bit RSA private key
..............................+++
................................................+++
writing new private key to '/root/k8s/easy-rsa-master/easyrsa3/pki/private/server.key'
-----
Using configuration from /root/k8s/easy-rsa-master/easyrsa3/openssl-1.0.cnf
Check that the request matches the signature
Signature ok
The Subject's Distinguished Name is as follows
commonName            :PRINTABLE:'server'
Certificate is to be certified until May 25 03:23:50 2027 GMT (3650 days)

Write out database with 1 new entries
Data Base Updated

• 拷贝pki/ca.crt、pki/issued/server.crt和pki/private/server.key至指定的目录

# mkdir /etc/kubernetes/pki
# cp pki/ca.crt pki/issued/server.crt pki/private/server.key /etc/kubernetes/pki/
# chmod 644 /etc/kubernetes/pki/*

配置kube-apiserver服务

参照以下内容编辑/etc/kubernetes/apiserver：

...
# Add your own!
KUBE_API_ARGS="--client-ca-file=/etc/kubernetes/pki/ca.crt --tls-cert-file=/etc/kubernetes/pki/server.crt --tls-private-key-file=/etc/kubernetes/pki/server.key"

配置kube-controller-manager服务

参照以下内容编辑/etc/kubernetes/controller-manager：

...
# Add your own!
KUBE_CONTROLLER_MANAGER_ARGS="--service_account_private_key_file=/etc/kubernetes/pki/server.key --root-ca-file=/etc/kubernetes/pki/ca.crt"

删除旧secrets资源

# kubectl get secrets --all-namespaces
NAMESPACE     NAME                  TYPE                                  DATA      AGE
default       default-token-s1vfh   kubernetes.io/service-account-token   2         5m
kube-system   default-token-jct68   kubernetes.io/service-account-token   2         4m

# systemctl stop kube-controller-manager

# kubectl delete secret default-token-s1vfh
secret "default-token-s1vfh" deleted

# kubectl delete secret default-token-jct68 --namespace=kube-system
secret "default-token-jct68" deleted

重新启动kube-apiserver和kube-controller-manager服务

# systemctl restart kube-apiserver
# systemctl start  kube-controller-manager

检查新创建的secret是否包含根证书

# kubectl get secrets --all-namespaces
NAMESPACE     NAME                  TYPE                                  DATA      AGE
default       default-token-tv69r   kubernetes.io/service-account-token   3         3s
kube-system   default-token-27w5m   kubernetes.io/service-account-token   3         3s

# kubectl describe secret default-token-27w5m --namespace=kube-system
Name:       default-token-27w5m
Namespace:  kube-system
Labels:     
Annotations:    kubernetes.io/service-account.name=default
        kubernetes.io/service-account.uid=80bc5d75-41e9-11e7-b90e-000c29f6f813

Type:   kubernetes.io/service-account-token

Data
====
ca.crt:     1233 bytes
namespace:  11 bytes
token:      eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJrdWJlcm5ldGVzL3NlcnZpY2VhY2NvdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9uYW1lc3BhY2UiOiJrdWJlLXN5c3RlbSIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VjcmV0Lm5hbWUiOiJkZWZhdWx0LXRva2VuLTI3dzVtIiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9zZXJ2aWNlLWFjY291bnQubmFtZSI6ImRlZmF1bHQiLCJrdWJlcm5ldGVzLmlvL3NlcnZpY2VhY2NvdW50L3NlcnZpY2UtYWNjb3VudC51aWQiOiI4MGJjNWQ3NS00MWU5LTExZTctYjkwZS0wMDBjMjlmNmY4MTMiLCJzdWIiOiJzeXN0ZW06c2VydmljZWFjY291bnQ6a3ViZS1zeXN0ZW06ZGVmYXVsdCJ9.CCxdtFRagtEo2eiPZgkiHjLkGDgbvt7VWe2WZGsLKeh_7Z2t-bUawwXGYxgd0MT_lG2HJbmHRTUb57Zw1MGRMZ-u4dBx_J9hXztnrdWcOh8_L_stk64gFQXjXpuZee1ltDksm7pTXtCnG1x8zBBxoZVi0jadPDMC_HP2OzvJXHrUPbCb58PBIqjRjbuJUQgM_hooDoJryK_0wYOd8TWOKUJMqQdJwTozFciDcGE__F3BchgHqfO9064f3ki1qSrZsnTImTpCYsUu4sy1fbL7X-3mVFWNNbsIvscFnBWP1Poj2M_hgqG_e4VCXL6vv61ll1LytWUwqPxosk1Djk7rvQ

可以看到新创建的secret资源已包含ca.crt。

重新创建Dashboard Pod

# kubectl delete -f kube-dashboard-no-rbac.yaml
deployment "kubernetes-dashboard" deleted

# kubectl create -f kube-dashboard-no-rbac.yaml
deployment "kubernetes-dashboard" created
service "kubernetes-dashboard" created

# kubectl get pods --namespace=kube-system -o wide
NAME                                    READY     STATUS    RESTARTS   AGE       IP             NODE
kubernetes-dashboard-3951142596-qxsb0   1/1       Running   0          1m        172.30.103.4   kube-node3

此次，kubernetes-dashboard-3951142596-qxsb0状态为Running，表示该Pod正常运行。

打开浏览器，输入http://192.168.120.121:8080/ui，即可访问可视化界面：

至此，完成Kuberbetes Dashboard的搭建。