1、Pod的健康检查，也叫做探针，探针的种类有两种。
答：1）、livenessProbe，健康状态检查，周期性检查服务是否存活，检查结果失败，将重启容器。
2）、readinessProbe，可用性检查，周期性检查服务是否可用，不可用将从service的endpoints中移除。
2、探针的检测方法。
答：1）、exec，执行一段命令。
2）、httpGet，检测某个http请求的返回状态码。
3）、tcpSocket，测试某个端口是否能够连接。
3、创建exec的探针检测方法。
1 [root@k8s-master health]# vim pod_nginx_exec.yaml
1 apiVersion: v1 2 kind: Pod 3 metadata: 4 name: exec 5 spec: 6 containers: 7 - name: nginx 8 image: 192.168.110.133:5000/nginx:1.13 9 ports: 10 - containerPort: 80 11 args: 12 - /bin/sh 13 - -c 14 - touch /tmp/healthy; sleep 30; rm -rf /tmp/healthy; sleep 600 15 livenessProbe: 16 exec: 17 command: 18 - cat 19 - /tmp/healthy 20 initialDelaySeconds: 5 21 periodSeconds: 5 22
开始创建这个Pod，操作如下所示：
1 [root@k8s-master health]# kubectl create -f pod_nginx_exec.yaml 2 pod "exec" created 3 [root@k8s-master health]# kubectl get all -o wide 4 NAME DESIRED CURRENT READY AGE CONTAINER(S) IMAGE(S) SELECTOR 5 rc/mysql 1 1 1 1h mysql 192.168.110.133:5000/mysql:5.7.30 app=mysql 6 rc/myweb 1 1 1 1h myweb 192.168.110.133:5000/tomcat:latest app=myweb 7 8 NAME CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR 9 svc/kubernetes 10.254.0.1443/TCP 13d 10 svc/mysql 10.254.207.238 3306/TCP 1h app=mysql 11 svc/myweb 10.254.29.22 8080:30008/TCP 1h app=myweb 12 13 NAME READY STATUS RESTARTS AGE IP NODE 14 po/busybox2 1/1 Running 1 35m 172.16.16.3 k8s-master 15 po/exec 1/1 Running 0 15s 172.16.59.4 k8s-node2 16 po/mysql-lmx4s 1/1 Running 0 1h 172.16.59.2 k8s-node2 17 po/myweb-hsdwn 1/1 Running 0 1h 172.16.32.3 k8s-node3 18 [root@k8s-master health]#
可用通过详细查看Pod的启动状况。
1 [root@k8s-master health]# kubectl describe pod exec 2 Name: exec 3 Namespace: default 4 Node: k8s-node2/192.168.110.134 5 Start Time: Thu, 18 Jun 2020 16:36:16 +0800 6 Labels:7 Status: Running 8 IP: 172.16.59.4 9 Controllers: 10 Containers: 11 nginx: 12 Container ID: docker://54e08c980bd1ad05c756de4de50b0f61c448a01b7ffcda2b97dc8b5c3a6e2749 13 Image: 192.168.110.133:5000/nginx:1.13 14 Image ID: docker-pullable://192.168.110.133:5000/nginx@sha256:e4f0474a75c510f40b37b6b7dc2516241ffa8bde5a442bde3d372c9519c84d90 15 Port: 80/TCP 16 Args: 17 /bin/sh 18 -c 19 touch /tmp/healthy; sleep 30; rm -rf /tmp/healthy; sleep 600 20 State: Running 21 Started: Thu, 18 Jun 2020 16:37:32 +0800 22 Last State: Terminated 23 Reason: Error 24 Exit Code: 137 25 Started: Thu, 18 Jun 2020 16:36:18 +0800 26 Finished: Thu, 18 Jun 2020 16:37:32 +0800 27 Ready: True 28 Restart Count: 1 29 Liveness: exec [cat /tmp/healthy] delay=5s timeout=1s period=5s #success=1 #failure=3 30 Volume Mounts: 31 Environment Variables: 32 Conditions: 33 Type Status 34 Initialized True 35 Ready True 36 PodScheduled True 37 No volumes. 38 QoS Class: BestEffort 39 Tolerations: 40 Events: 41 FirstSeen LastSeen Count From SubObjectPath Type Reason Message 42 --------- -------- ----- ---- ------------- -------- ------ ------- 43 1m 1m 1 {default-scheduler } Normal Scheduled Successfully assigned exec to k8s-node2 44 1m 1m 1 {kubelet k8s-node2} spec.containers{nginx} Normal Created Created container with docker id ff079949b2cb; Security:[seccomp=unconfined] 45 1m 1m 1 {kubelet k8s-node2} spec.containers{nginx} Normal Started Started container with docker id ff079949b2cb 46 1m 1m 3 {kubelet k8s-node2} spec.containers{nginx} Warning Unhealthy Liveness probe failed: cat: /tmp/healthy: No such file or directory 47 48 1m 33s 2 {kubelet k8s-node2} spec.containers{nginx} Normal Pulled Container image "192.168.110.133:5000/nginx:1.13" already present on machine 49 33s 33s 1 {kubelet k8s-node2} spec.containers{nginx} Normal Killing Killing container with docker id ff079949b2cb: pod "exec_default(c69fbd14-b13e-11ea-80b4-000c2919d52d)" container "nginx" is unhealthy, it will be killed and re-created. 50 33s 33s 1 {kubelet k8s-node2} spec.containers{nginx} Normal Created Created container with docker id 54e08c980bd1; Security:[seccomp=unconfined] 51 33s 33s 1 {kubelet k8s-node2} spec.containers{nginx} Normal Started Started container with docker id 54e08c980bd1 52 [root@k8s-master health]# kubectl get all -o wide 53 NAME DESIRED CURRENT READY AGE CONTAINER(S) IMAGE(S) SELECTOR 54 rc/mysql 1 1 1 1h mysql 192.168.110.133:5000/mysql:5.7.30 app=mysql 55 rc/myweb 1 1 1 1h myweb 192.168.110.133:5000/tomcat:latest app=myweb 56 57 NAME CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR 58 svc/kubernetes 10.254.0.1 443/TCP 13d 59 svc/mysql 10.254.207.238 3306/TCP 1h app=mysql 60 svc/myweb 10.254.29.22 8080:30008/TCP 1h app=myweb 61 62 NAME READY STATUS RESTARTS AGE IP NODE 63 po/busybox2 1/1 Running 1 37m 172.16.16.3 k8s-master 64 po/exec 1/1 Running 1 2m 172.16.59.4 k8s-node2 65 po/mysql-lmx4s 1/1 Running 0 1h 172.16.59.2 k8s-node2 66 po/myweb-hsdwn 1/1 Running 0 1h 172.16.32.3 k8s-node3 67 [root@k8s-master health]#
pod重启的次数，可以看到这个Pod的重启了多少次了。
4、创建httpGet的探针检测方法，liveness探针的httpGet使用。
1 [root@k8s-master health]# vim nginx_pod_httpGet.yaml
具体内容，如下所示：
1 apiVersion: v1 2 kind: Pod 3 metadata: 4 name: httpget 5 spec: 6 containers: 7 - name: nginx 8 image: 192.168.110.133:5000/nginx:1.13 9 ports: 10 - containerPort: 80 11 livenessProbe: 12 httpGet: 13 path: /index.html 14 port: 80 15 initialDelaySeconds: 3 16 periodSeconds: 3 17
创建httpGet的探针检测方法，liveness探针的httpGet使用。
1 [root@k8s-master health]# kubectl get all -o wide 2 NAME DESIRED CURRENT READY AGE CONTAINER(S) IMAGE(S) SELECTOR 3 rc/mysql 1 1 1 1h mysql 192.168.110.133:5000/mysql:5.7.30 app=mysql 4 rc/myweb 1 1 1 1h myweb 192.168.110.133:5000/tomcat:latest app=myweb 5 6 NAME CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR 7 svc/kubernetes 10.254.0.1443/TCP 13d 8 svc/mysql 10.254.207.238 3306/TCP 1h app=mysql 9 svc/myweb 10.254.29.22 8080:30008/TCP 1h app=myweb 10 11 NAME READY STATUS RESTARTS AGE IP NODE 12 po/busybox2 1/1 Running 1 46m 172.16.16.3 k8s-master 13 po/exec 0/1 CrashLoopBackOff 6 11m 172.16.59.4 k8s-node2 14 po/httpget 1/1 Running 0 22s 172.16.16.4 k8s-master 15 po/mysql-lmx4s 1/1 Running 0 1h 172.16.59.2 k8s-node2 16 po/myweb-hsdwn 1/1 Running 0 1h 172.16.32.3 k8s-node3 17 [root@k8s-master health]# kubectl describe pod httpget 18 Name: httpget 19 Namespace: default 20 Node: k8s-master/192.168.110.133 21 Start Time: Thu, 18 Jun 2020 16:47:03 +0800 22 Labels: 23 Status: Running 24 IP: 172.16.16.4 25 Controllers: 26 Containers: 27 nginx: 28 Container ID: docker://f9e0ea6a16a26d3fca57b1e1b521f642f89eb05e6951a05830933e68dce1253a 29 Image: 192.168.110.133:5000/nginx:1.13 30 Image ID: docker-pullable://docker.io/nginx@sha256:b1d09e9718890e6ebbbd2bc319ef1611559e30ce1b6f56b2e3b479d9da51dc35 31 Port: 80/TCP 32 State: Running 33 Started: Thu, 18 Jun 2020 16:47:04 +0800 34 Ready: True 35 Restart Count: 0 36 Liveness: http-get http://:80/index.html delay=3s timeout=1s period=3s #success=1 #failure=3 37 Volume Mounts: 38 Environment Variables: 39 Conditions: 40 Type Status 41 Initialized True 42 Ready True 43 PodScheduled True 44 No volumes. 45 QoS Class: BestEffort 46 Tolerations: 47 Events: 48 FirstSeen LastSeen Count From SubObjectPath Type Reason Message 49 --------- -------- ----- ---- ------------- -------- ------ ------- 50 1m 1m 1 {default-scheduler } Normal Scheduled Successfully assigned httpget to k8s-master 51 1m 1m 1 {kubelet k8s-master} spec.containers{nginx} Normal Pulled Container image "192.168.110.133:5000/nginx:1.13" already present on machine 52 1m 1m 1 {kubelet k8s-master} spec.containers{nginx} Normal Created Created container with docker id f9e0ea6a16a2; Security:[seccomp=unconfined] 53 1m 1m 1 {kubelet k8s-master} spec.containers{nginx} Normal Started Started container with docker id f9e0ea6a16a2 54 [root@k8s-master health]#
默认，这个首页index.html是可以访问到的，这里也可以让它访问不到的。
1 [root@k8s-master health]# kubectl exec -it httpget bash 2 root@httpget:/# cd /usr/share/html 3 bash: cd: /usr/share/html: No such file or directory 4 root@httpget:/# cd /usr/share/nginx/html 5 root@httpget:/usr/share/nginx/html# ls 6 50x.html index.html 7 root@httpget:/usr/share/nginx/html# mv index.html /tmp/ 8 root@httpget:/usr/share/nginx/html# exit 9 exit 10 [root@k8s-master health]# kubectl get pod httpget 11 NAME READY STATUS RESTARTS AGE 12 httpget 1/1 Running 1 3m 13 [root@k8s-master health]# kubectl describe pod httpget 14 Name: httpget 15 Namespace: default 16 Node: k8s-master/192.168.110.133 17 Start Time: Thu, 18 Jun 2020 16:47:03 +0800 18 Labels:19 Status: Running 20 IP: 172.16.16.4 21 Controllers: 22 Containers: 23 nginx: 24 Container ID: docker://7bb1ef7b33b24e16aad18053b3238077baac8e4b507ba85a6943dc9e9ae3ea29 25 Image: 192.168.110.133:5000/nginx:1.13 26 Image ID: docker-pullable://docker.io/nginx@sha256:b1d09e9718890e6ebbbd2bc319ef1611559e30ce1b6f56b2e3b479d9da51dc35 27 Port: 80/TCP 28 State: Running 29 Started: Thu, 18 Jun 2020 16:50:07 +0800 30 Last State: Terminated 31 Reason: Completed 32 Exit Code: 0 33 Started: Thu, 18 Jun 2020 16:47:04 +0800 34 Finished: Thu, 18 Jun 2020 16:50:06 +0800 35 Ready: True 36 Restart Count: 1 37 Liveness: http-get http://:80/index.html delay=3s timeout=1s period=3s #success=1 #failure=3 38 Volume Mounts: 39 Environment Variables: 40 Conditions: 41 Type Status 42 Initialized True 43 Ready True 44 PodScheduled True 45 No volumes. 46 QoS Class: BestEffort 47 Tolerations: 48 Events: 49 FirstSeen LastSeen Count From SubObjectPath Type Reason Message 50 --------- -------- ----- ---- ------------- -------- ------ ------- 51 3m 3m 1 {default-scheduler } Normal Scheduled Successfully assigned httpget to k8s-master 52 3m 3m 1 {kubelet k8s-master} spec.containers{nginx} Normal Created Created container with docker id f9e0ea6a16a2; Security:[seccomp=unconfined] 53 3m 3m 1 {kubelet k8s-master} spec.containers{nginx} Normal Started Started container with docker id f9e0ea6a16a2 54 3m 13s 2 {kubelet k8s-master} spec.containers{nginx} Normal Pulled Container image "192.168.110.133:5000/nginx:1.13" already present on machine 55 19s 13s 3 {kubelet k8s-master} spec.containers{nginx} Warning Unhealthy Liveness probe failed: HTTP probe failed with statuscode: 404 56 13s 13s 1 {kubelet k8s-master} spec.containers{nginx} Normal Killing Killing container with docker id f9e0ea6a16a2: pod "httpget_default(4857d09b-b140-11ea-80b4-000c2919d52d)" container "nginx" is unhealthy, it will be killed and re-created. 57 13s 13s 1 {kubelet k8s-master} spec.containers{nginx} Normal Created Created container with docker id 7bb1ef7b33b2; Security:[seccomp=unconfined] 58 12s 12s 1 {kubelet k8s-master} spec.containers{nginx} Normal Started Started container with docker id 7bb1ef7b33b2 59 [root@k8s-master health]# kubectl describe pod httpget 60 Name: httpget 61 Namespace: default 62 Node: k8s-master/192.168.110.133 63 Start Time: Thu, 18 Jun 2020 16:47:03 +0800 64 Labels: 65 Status: Running 66 IP: 172.16.16.4 67 Controllers: 68 Containers: 69 nginx: 70 Container ID: docker://7bb1ef7b33b24e16aad18053b3238077baac8e4b507ba85a6943dc9e9ae3ea29 71 Image: 192.168.110.133:5000/nginx:1.13 72 Image ID: docker-pullable://docker.io/nginx@sha256:b1d09e9718890e6ebbbd2bc319ef1611559e30ce1b6f56b2e3b479d9da51dc35 73 Port: 80/TCP 74 State: Running 75 Started: Thu, 18 Jun 2020 16:50:07 +0800 76 Last State: Terminated 77 Reason: Completed 78 Exit Code: 0 79 Started: Thu, 18 Jun 2020 16:47:04 +0800 80 Finished: Thu, 18 Jun 2020 16:50:06 +0800 81 Ready: True 82 Restart Count: 1 83 Liveness: http-get http://:80/index.html delay=3s timeout=1s period=3s #success=1 #failure=3 84 Volume Mounts: 85 Environment Variables: 86 Conditions: 87 Type Status 88 Initialized True 89 Ready True 90 PodScheduled True 91 No volumes. 92 QoS Class: BestEffort 93 Tolerations: 94 Events: 95 FirstSeen LastSeen Count From SubObjectPath Type Reason Message 96 --------- -------- ----- ---- ------------- -------- ------ ------- 97 3m 3m 1 {default-scheduler } Normal Scheduled Successfully assigned httpget to k8s-master 98 3m 3m 1 {kubelet k8s-master} spec.containers{nginx} Normal Created Created container with docker id f9e0ea6a16a2; Security:[seccomp=unconfined] 99 3m 3m 1 {kubelet k8s-master} spec.containers{nginx} Normal Started Started container with docker id f9e0ea6a16a2 100 3m 18s 2 {kubelet k8s-master} spec.containers{nginx} Normal Pulled Container image "192.168.110.133:5000/nginx:1.13" already present on machine 101 24s 18s 3 {kubelet k8s-master} spec.containers{nginx} Warning Unhealthy Liveness probe failed: HTTP probe failed with statuscode: 404 102 18s 18s 1 {kubelet k8s-master} spec.containers{nginx} Normal Killing Killing container with docker id f9e0ea6a16a2: pod "httpget_default(4857d09b-b140-11ea-80b4-000c2919d52d)" container "nginx" is unhealthy, it will be killed and re-created. 103 18s 18s 1 {kubelet k8s-master} spec.containers{nginx} Normal Created Created container with docker id 7bb1ef7b33b2; Security:[seccomp=unconfined] 104 17s 17s 1 {kubelet k8s-master} spec.containers{nginx} Normal Started Started container with docker id 7bb1ef7b33b2 105 [root@k8s-master health]#
5、liveness探针的tcpSocket使用。
1 [root@k8s-master health]# vim nginx_pod_tcpSocket.yaml
使用tcpSocket监控的是80的端口，配置文件的内容如下所示：
1 apiVersion: v1 2 kind: Pod 3 metadata: 4 name: tcpsocket 5 spec: 6 containers: 7 - name: nginx 8 image: 192.168.110.133:5000/nginx:1.13 9 ports: 10 - containerPort: 80 11 livenessProbe: 12 tcpSocket: 13 port: 80 14 initialDelaySeconds: 3 15 periodSeconds: 3 16
创建，如下所示：
1 [root@k8s-master health]# kubectl create -f nginx_pod_tcpSocket.yaml 2 pod "tcpsocket" created 3 [root@k8s-master health]# kubectl get all -o wide 4 NAME CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR 5 svc/kubernetes 10.254.0.1443/TCP 13d 6 7 NAME READY STATUS RESTARTS AGE IP NODE 8 po/busybox2 1/1 Running 1 54m 172.16.16.3 k8s-master 9 po/exec 0/1 CrashLoopBackOff 7 18m 172.16.59.4 k8s-node2 10 po/httpget 1/1 Running 1 8m 172.16.16.4 k8s-master 11 po/tcpsocket 1/1 Running 0 6s 172.16.59.2 k8s-node2
6、readiness探针的httpGet使用。
1 [root@k8s-master health]# vim nginx_rc_readiness.yaml
具体内容，如下所示：
需要访问qiangge.html，如果不存在就一直加入不到svc中。
1 apiVersion: v1 2 kind: ReplicationController 3 metadata: 4 name: readiness 5 spec: 6 replicas: 2 7 selector: 8 app: readiness 9 template: 10 metadata: 11 labels: 12 app: readiness 13 spec: 14 containers: 15 - name: readiness 16 image: 192.168.110.133:5000/nginx:1.13 17 ports: 18 - containerPort: 80 19 readinessProbe: 20 httpGet: 21 path: /qiangge.html 22 port: 80 23 initialDelaySeconds: 3 24 periodSeconds: 3
创建完rc之后，需要创建一个svc的，这里使用命令创建，不再使用配置文件进行创建了。
1 [root@k8s-master health]# kubectl expose rc readiness --port=80 2 service "readiness" exposed 3 [root@k8s-master health]# kubectl describe svc readiness 4 Name: readiness 5 Namespace: default 6 Labels: app=readiness 7 Selector: app=readiness 8 Type: ClusterIP 9 IP: 10.254.218.107 10 Port:80/TCP 11 Endpoints: 12 Session Affinity: None 13 No events. 14 [root@k8s-master health]# kubectl get all -o wide 15 NAME DESIRED CURRENT READY AGE CONTAINER(S) IMAGE(S) SELECTOR 16 rc/readiness 2 2 0 2m readiness 192.168.110.133:5000/nginx:1.13 app=readiness 17 18 NAME CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR 19 svc/kubernetes 10.254.0.1 443/TCP 13d 20 svc/readiness 10.254.218.107 80/TCP 28s app=readiness 21 22 NAME READY STATUS RESTARTS AGE IP NODE 23 po/busybox2 1/1 Running 1 1h 172.16.16.3 k8s-master 24 po/exec 1/1 Running 11 28m 172.16.59.4 k8s-node2 25 po/httpget 1/1 Running 1 18m 172.16.16.4 k8s-master 26 po/readiness-66j6c 0/1 Running 0 2m 172.16.59.5 k8s-node2 27 po/readiness-l2v6z 0/1 Running 0 2m 172.16.32.3 k8s-node3 28 po/tcpsocket 1/1 Running 0 10m 172.16.59.2 k8s-node2 29 [root@k8s-master health]#
可以看到readiness已经启动了两个Pod了，但是后端节点里面是空的。此时可以创建一个指定的html文件，就可以了。
此时发现后端节点也不为空了，Pod也正常启动了，另外一个可以类似出来，就可以将两个Pod正常启动起来了。
1 [root@k8s-master health]# kubectl exec -it readiness-66j6c bash 2 root@readiness-66j6c:/# cd /usr/share/nginx/html/ 3 root@readiness-66j6c:/usr/share/nginx/html# echo hello nginx > qiangge.html 4 root@readiness-66j6c:/usr/share/nginx/html# cat qiangge.html 5 hello nginx 6 root@readiness-66j6c:/usr/share/nginx/html# exit 7 exit 8 [root@k8s-master health]# kubectl describe svc readiness 9 Name: readiness 10 Namespace: default 11 Labels: app=readiness 12 Selector: app=readiness 13 Type: ClusterIP 14 IP: 10.254.218.107 15 Port:80/TCP 16 Endpoints: 172.16.59.5:80 17 Session Affinity: None 18 No events. 19 [root@k8s-master health]# kubectl get all -o wide 20 NAME DESIRED CURRENT READY AGE CONTAINER(S) IMAGE(S) SELECTOR 21 rc/readiness 2 2 1 6m readiness 192.168.110.133:5000/nginx:1.13 app=readiness 22 23 NAME CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR 24 svc/kubernetes 10.254.0.1 443/TCP 13d 25 svc/readiness 10.254.218.107 80/TCP 4m app=readiness 26 27 NAME READY STATUS RESTARTS AGE IP NODE 28 po/busybox2 1/1 Running 1 1h 172.16.16.3 k8s-master 29 po/exec 0/1 CrashLoopBackOff 11 32m 172.16.59.4 k8s-node2 30 po/httpget 1/1 Running 1 21m 172.16.16.4 k8s-master 31 po/readiness-66j6c 1/1 Running 0 6m 172.16.59.5 k8s-node2 32 po/readiness-l2v6z 0/1 Running 0 6m 172.16.32.3 k8s-node3 33 po/tcpsocket 1/1 Running 0 13m 172.16.59.2 k8s-node2 34 [root@k8s-master health]#