Kubernetes Dashboard 是 k8s集群的一个 WEB UI管理工具,代码托管在 github 上,地址:https://github.com/kubernetes/dashboard
创建tls secret
通过https进行访问必需要使用证书和密钥,在Kubernetes中可以通过配置一个加密凭证(TLS secret)来提供。
这里只是拿来自己使用,创建一个自己签名的证书。
openssl req -x509 -nodes -days 3650 -newkey rsa:2048 -keyout ./tls.key -out ./tls.crt -subj "/CN=192.168.236.130"
将会产生两个文件tls.key和tls.crt,你可以改成自己的文件名或放在特定的目录下(如果你是为公共服务器创建的,请保证这个不会被别人访问到)。后面的192.168.126.130
是我的服务器IP地址,你可以改成自己的。
安装tls secret
下一步,将这两个文件的信息创建为一个Kubernetes的secret访问凭证,我将名称指定为 hongda-com-tls-secret
,这在后面的Ingress配置时将会用到。如果你修改了这个名字,注意后面的配置yaml文件也需要同步修改。
kubectl -n kube-system create secret tls hongda-com-tls-secret --key ./tls.key --cert ./tls.crt
查看:
kubectl get secret -n kube-system |grep hongda
hongda-com-tls-secret kubernetes.io/tls 2 43s
安装
kubernetes-dashboard.yaml:
image:
repository: k8s.gcr.io/kubernetes-dashboard-amd64
tag: v1.10.1
ingress:
enabled: true
hosts:
- k8s.hongda.com
annotations:
nginx.ingress.kubernetes.io/ssl-redirect: "false"
nginx.ingress.kubernetes.io/backend-protocol: "HTTPS"
tls:
- secretName: hongda-com-tls-secret
hosts:
- k8s.hongda.com
nodeSelector:
node-role.kubernetes.io/edge: ''
tolerations:
- key: node-role.kubernetes.io/master
operator: Exists
effect: NoSchedule
- key: node-role.kubernetes.io/master
operator: Exists
effect: PreferNoSchedule
rbac:
clusterAdminRole: true
相比默认配置,修改了以下配置项:
- ingress.enabled - 置为 true 开启 Ingress,用 Ingress 将 Kubernetes Dashboard 服务暴露出来,以便让我们浏览器能够访问
- ingress.annotations - 指定
ingress.class
为 nginx,让我们安装 Nginx Ingress Controller 来反向代理 Kubernetes Dashboard 服务;由于 Kubernetes Dashboard 后端服务是以 https 方式监听的,而 Nginx Ingress Controller 默认会以 HTTP 协议将请求转发给后端服务,用secure-backends
这个 annotation 来指示 Nginx Ingress Controller 以 HTTPS 协议将请求转发给后端服务 - ingress.hosts - 这里替换为证书配置的域名
- Ingress.tls - secretName 配置为 cert-manager 生成的免费证书所在的 Secret 资源名称,hosts 替换为证书配置的域名
- rbac.clusterAdminRole - 置为 true 让 dashboard 的权限够大,这样我们可以方便操作多个 namespace
命令安装:
helm install stable/kubernetes-dashboard \
-n kubernetes-dashboard \
--namespace kube-system \
-f kubernetes-dashboard.yaml
输出:
[root@master /]# helm install stable/kubernetes-dashboard -n kubernetes-dashboard --namespace kube-system -f kubernetes-dashboard.yaml
NAME: kubernetes-dashboard
LAST DEPLOYED: Tue Aug 6 16:11:37 2019
NAMESPACE: kube-system
STATUS: DEPLOYED
RESOURCES:
==> v1/Deployment
NAME READY UP-TO-DATE AVAILABLE AGE
kubernetes-dashboard 0/1 1 0
==> v1/Pod(related)
NAME READY STATUS RESTARTS AGE
kubernetes-dashboard-848b8dd798-gtddg 0/1 ContainerCreating 0
==> v1/Secret
NAME TYPE DATA AGE
kubernetes-dashboard Opaque 0
==> v1/Service
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
kubernetes-dashboard ClusterIP 10.108.244.10 443/TCP
==> v1/ServiceAccount
NAME SECRETS AGE
kubernetes-dashboard 1
==> v1beta1/ClusterRoleBinding
NAME AGE
kubernetes-dashboard
==> v1beta1/Ingress
NAME HOSTS ADDRESS PORTS AGE
kubernetes-dashboard k8s.hongda.com 80, 443
NOTES:
*********************************************************************************
*** PLEASE BE PATIENT: kubernetes-dashboard may take a few minutes to install ***
*********************************************************************************
From outside the cluster, the server URL(s) are:
https://k8s.hongda.com
查看pods:
[root@master /]# kubectl get pods -n kube-system -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
coredns-5c98db65d4-gts57 1/1 Running 1 3d6h 10.244.2.2 slaver2
coredns-5c98db65d4-qhwrw 1/1 Running 1 3d6h 10.244.1.2 slaver1
etcd-master 1/1 Running 2 3d6h 18.16.202.163 master
kube-apiserver-master 1/1 Running 2 3d6h 18.16.202.163 master
kube-controller-manager-master 1/1 Running 6 3d6h 18.16.202.163 master
kube-flannel-ds-amd64-2lwl8 1/1 Running 0 3d1h 18.16.202.227 slaver1
kube-flannel-ds-amd64-9bjck 1/1 Running 0 3d1h 18.16.202.95 slaver2
kube-flannel-ds-amd64-gxxqg 1/1 Running 0 3d1h 18.16.202.163 master
kube-proxy-8cwj4 1/1 Running 0 107m 18.16.202.163 master
kube-proxy-j9zpz 1/1 Running 0 107m 18.16.202.227 slaver1
kube-proxy-vfgjv 1/1 Running 0 107m 18.16.202.95 slaver2
kube-scheduler-master 1/1 Running 6 3d6h 18.16.202.163 master
kubernetes-dashboard-64f97ccb4f-nbpkx 0/1 ImagePullBackOff 0 33m 10.244.0.4 master
tiller-deploy-6787c946f8-6b5tv 1/1 Running 0 44m 10.244.1.4 slaver1
异常问题
查看线上版本:
[root@master /]# helm search kubernetes-dashboard
NAME CHART VERSION APP VERSION DESCRIPTION
stable/kubernetes-dashboard 0.6.0 1.8.3 General-purpose web UI for Kubernetes clusters
应该是版本不一致,阿里云里最新版本为1.8.3
,而helm安装配置版本为1.10.1
,所以导致没有拉取到镜像
添加新的仓库源
[root@master /]# helm repo add stable http://mirror.azure.cn/kubernetes/charts/
"stable" has been added to your repositories
[root@master /]# helm search kubernetes-dashboard
NAME CHART VERSION APP VERSION DESCRIPTION
stable/kubernetes-dashboard 1.8.0 1.10.1 General-purpose web UI for Kubernetes clusters
更换仓库以后,再次安装,还是一样的问题,查看
[root@master /]# kubectl get namespace
NAME STATUS AGE
default Active 3d8h
ingress-nginx Active 152m
kube-node-lease Active 3d8h
kube-public Active 3d8h
kube-system Active 3d8h
[root@master /]# kubectl describe pod kubernetes-dashboard-7ffdf885d6-t4htt -n kube-system
Name: kubernetes-dashboard-7ffdf885d6-t4htt
Namespace: kube-system
Priority: 0
Node: master/18.16.202.163
Start Time: Wed, 31 Jul 2019 16:46:40 +0800
Labels: app=kubernetes-dashboard
kubernetes.io/cluster-service=true
pod-template-hash=7ffdf885d6
release=kubernetes-dashboard
Annotations:
Status: Pending
IP: 10.244.0.20
Controlled By: ReplicaSet/kubernetes-dashboard-7ffdf885d6
Containers:
kubernetes-dashboard:
Container ID:
Image: k8s.gcr.io/kubernetes-dashboard-amd64:v1.8.3
Image ID:
Port: 8443/TCP
Host Port: 0/TCP
Args:
--auto-generate-certificates
State: Waiting
Reason: ImagePullBackOff
Ready: False
Restart Count: 0
Limits:
cpu: 100m
memory: 50Mi
Requests:
cpu: 100m
memory: 50Mi
Liveness: http-get https://:8443/ delay=30s timeout=30s period=10s #success=1 #failure=3
Environment:
Mounts:
/certs from kubernetes-dashboard-certs (rw)
/tmp from tmp-volume (rw)
/var/run/secrets/kubernetes.io/serviceaccount from kubernetes-dashboard-token-pph4g (ro)
Conditions:
Type Status
Initialized True
Ready False
ContainersReady False
PodScheduled True
Volumes:
kubernetes-dashboard-certs:
Type: Secret (a volume populated by a Secret)
SecretName: kubernetes-dashboard
Optional: false
tmp-volume:
Type: EmptyDir (a temporary directory that shares a pod's lifetime)
Medium:
SizeLimit:
kubernetes-dashboard-token-pph4g:
Type: Secret (a volume populated by a Secret)
SecretName: kubernetes-dashboard-token-pph4g
Optional: false
QoS Class: Guaranteed
Node-Selectors: node-role.kubernetes.io/edge=
Tolerations: node-role.kubernetes.io/master:NoSchedule
node-role.kubernetes.io/master:PreferNoSchedule
node.kubernetes.io/not-ready:NoExecute for 300s
node.kubernetes.io/unreachable:NoExecute for 300s
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Scheduled 3m47s default-scheduler Successfully assigned kube-system/kubernetes-dashboard-7ffdf885d6-t4htt to master
Normal Pulling 89s (x4 over 3m45s) kubelet, master Pulling image "k8s.gcr.io/kubernetes-dashboard-amd64:v1.8.3"
Warning Failed 74s (x4 over 3m30s) kubelet, master Failed to pull image "k8s.gcr.io/kubernetes-dashboard-amd64:v1.8.3": rpc error: code = Unknown desc = Error response from daemon: Get https://k8s.gcr.io/v2/: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)
Warning Failed 74s (x4 over 3m30s) kubelet, master Error: ErrImagePull
Normal BackOff 61s (x6 over 3m30s) kubelet, master Back-off pulling image "k8s.gcr.io/kubernetes-dashboard-amd64:v1.8.3"
Warning Failed 46s (x7 over 3m30s) kubelet, master Error: ImagePullBackOff
明显是特么的拉取的k8s.gcr.io
域名下面的,拉取不到。
好吧,我还是拉取不到。
解决问题
从Docker Hub
中拉取一个相同版本的,替换
拉取
docker pull sacred02/kubernetes-dashboard-amd64:v1.10.1
替换
docker tag sacred02/kubernetes-dashboard-amd64:v1.10.1 k8s.gcr.io/kubernetes-dashboard-amd64:v1.10.1
删除
docker rmi sacred02/kubernetes-dashboard-amd64:v1.10.1
再次使用helm安装
helm install stable/kubernetes-dashboard -n kubernetes-dashboard --namespace kube-system -f kubernetes-dashboard.yaml
查看
[root@master /]# helm ls
NAME REVISION UPDATED STATUS CHART APP VERSION NAMESPACE
kubernetes-dashboard 1 Wed Jul 31 17:11:35 2019 DEPLOYED kubernetes-dashboard-1.8.0 1.10.1 kube-system
nginx-ingress 1 Wed Jul 31 13:59:14 2019 DEPLOYED nginx-ingress-1.11.5 0.25.0 ingress-nginx
查看po,svc:
[root@master /]# kubectl get po,svc --all-namespaces -o wide
NAMESPACE NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
default pod/curl-6bf6db5c4f-vhsqc 1/1 Running 1 10d 10.244.2.3 slaver2
ingress-nginx pod/nginx-ingress-controller-b89575c7f-2xtkk 1/1 Running 0 26m 18.16.202.163 master
ingress-nginx pod/nginx-ingress-default-backend-7b8b45bd49-g4mbz 1/1 Running 0 26m 10.244.0.23 master
kube-system pod/coredns-5c98db65d4-gts57 1/1 Running 7 11d 10.244.2.2 slaver2
kube-system pod/coredns-5c98db65d4-qhwrw 1/1 Running 6 11d 10.244.1.2 slaver1
kube-system pod/etcd-master 1/1 Running 4 11d 18.16.202.163 master
kube-system pod/kube-apiserver-master 1/1 Running 4 11d 18.16.202.163 master
kube-system pod/kube-controller-manager-master 1/1 Running 8 11d 18.16.202.163 master
kube-system pod/kube-flannel-ds-amd64-2lwl8 1/1 Running 0 11d 18.16.202.227 slaver1
kube-system pod/kube-flannel-ds-amd64-9bjck 1/1 Running 0 11d 18.16.202.95 slaver2
kube-system pod/kube-flannel-ds-amd64-gxxqg 1/1 Running 3 11d 18.16.202.163 master
kube-system pod/kube-proxy-8cwj4 1/1 Running 3 8d 18.16.202.163 master
kube-system pod/kube-proxy-j9zpz 1/1 Running 0 8d 18.16.202.227 slaver1
kube-system pod/kube-proxy-vfgjv 1/1 Running 0 8d 18.16.202.95 slaver2
kube-system pod/kube-scheduler-master 1/1 Running 8 11d 18.16.202.163 master
kube-system pod/kubernetes-dashboard-848b8dd798-gtddg 1/1 Running 0 40s 10.244.0.24 master
kube-system pod/tiller-deploy-6787c946f8-6b5tv 1/1 Running 0 8d 10.244.1.4 slaver1
NAMESPACE NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR
default service/kubernetes ClusterIP 10.96.0.1 443/TCP 11d
ingress-nginx service/nginx-ingress-controller LoadBalancer 10.111.25.193 80:31577/TCP,443:31246/TCP 26m app=nginx-ingress,component=controller,release=nginx-ingress
ingress-nginx service/nginx-ingress-default-backend ClusterIP 10.106.126.222 80/TCP 26m app=nginx-ingress,component=default-backend,release=nginx-ingress
kube-system service/kube-dns ClusterIP 10.96.0.10 53/UDP,53/TCP,9153/TCP 11d k8s-app=kube-dns
kube-system service/kubernetes-dashboard ClusterIP 10.108.244.10 443/TCP 40s app=kubernetes-dashboard,release=kubernetes-dashboard
kube-system service/tiller-deploy ClusterIP 10.98.116.74 44134/TCP 8d app=helm,name=tiller
token查看
[root@master /]# kubectl -n kube-system get secret | grep kubernetes-dashboard-token
kubernetes-dashboard-token-4v624 kubernetes.io/service-account-token 3 5m42s
[root@master /]# kubectl describe -n kube-system secret/kubernetes-dashboard-token-4v624
Name: kubernetes-dashboard-token-4v624
Namespace: kube-system
Labels:
Annotations: kubernetes.io/service-account.name: kubernetes-dashboard
kubernetes.io/service-account.uid: 6688cc3b-5f28-4e38-a37a-67c0927752ab
Type: kubernetes.io/service-account-token
Data
====
ca.crt: 1025 bytes
namespace: 11 bytes
token: eyJhbGciOiJSUzI1NiIsImtpZCI6IiJ9.eyJpc3MiOiJrdWJlcm5ldGVzL3NlcnZpY2VhY2NvdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9uYW1lc3BhY2UiOiJrdWJlLXN5c3RlbSIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VjcmV0Lm5hbWUiOiJrdWJlcm5ldGVzLWRhc2hib2FyZC10b2tlbi00djYyNCIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VydmljZS1hY2NvdW50Lm5hbWUiOiJrdWJlcm5ldGVzLWRhc2hib2FyZCIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VydmljZS1hY2NvdW50LnVpZCI6IjY2ODhjYzNiLTVmMjgtNGUzOC1hMzdhLTY3YzA5Mjc3NTJhYiIsInN1YiI6InN5c3RlbTpzZXJ2aWNlYWNjb3VudDprdWJlLXN5c3RlbTprdWJlcm5ldGVzLWRhc2hib2FyZCJ9.Wq6xvzLSJNnt9Zg9u5J-85RB0-Slf6HMFfHzNwDGJDn3Yc2lfxL88YXi0ForX4Q9F0v96nt_GNKOm6DB8FGoKR3cALeWpeuoXSSY_ryY8tj6KFN1mrOlvVnRRgsk_lReOxLZexvR58OQ7N04pDrZ6Okr3PDB22i-31xPaVPBt6BhZU5ee6VZyXr7y3pj8VAJSki7tnr7ZRlG6WJizrMf25sZ9xdznwcGJ7yGz2gD3moYhNKQa5KPwcLOGTfg3GuLUNoQjdz5wUmvx4X2YMhfj6Fx7I3mZzr9whrfhO2PWuNtFheaKscSg2UyIPH5Zav9WTSzXxDedORh8BjX3cUJcQ
查看k8s.hongda.com
[root@master /]# ping k8s.hongda.com
PING k8s.hongda.com (13.209.58.121) 56(84) bytes of data.
From 18.16.202.169 (18.16.202.169): icmp_seq=2 Redirect Network(New nexthop: 18.16.202.1 (18.16.202.1))
From 18.16.202.169 (18.16.202.169): icmp_seq=3 Redirect Network(New nexthop: 18.16.202.1 (18.16.202.1))
^C
--- k8s.hongda.com ping statistics ---
3 packets transmitted, 0 received, 100% packet loss, time 2002ms
参考:
使用kubeadm安装Kubernetes 1.15
利用Helm一键部署Kubernetes Dashboard并启用免费HTTPS
Kubernetes dashboard 通过 Ingress 提供HTTPS访问