容器探针

探针是由 kubelet对容器执行的定期诊断。要执行诊断, kubelet 调用由容器实现的 Handler 。有三种类型的处理程序:

  • ExecAction :在容器内执行指定命令。如果命令退出时返回码为 0 则认为诊断成功。
  • TCPSocketAction :对指定端口上的容器的 IP 地址进行 TCP 检查。如果端口打开,则诊断被认为是成功的。
  • HTTPGetAction :对指定的端口和路径上的容器的 IP 地址执行 HTTP Get 请求。如果响应的状态码大于等于 200 且小于 400 ,则诊断被认为是成功的

每次探测都将获得以下三种结果之一:

  • 成功:容器通过了诊断。
  • 失败:容器未通过诊断。
  • 未知:诊断失败,因此不会采取任何行动

探针的方式

  • livenessProbe :指示容器是否正在运行。如果存活探测失败,则 kubelet 会杀死容器,并且容器将受到其 重启策略 的影响。如果容器不提供存活探针,则默认状态为 Success
  • readinessProbe :指示容器是否准备好服务请求。如果就绪探测失败,端点控制器将从与 Pod 匹配的所有 Service 的端点中删除该 Pod 的 IP 地址。初始延迟之前的就绪状态默认为 Failure 。如果容器不提供就绪探针,则默认状态为 Success。

测试

检测探针 - 就绪检测

read.yaml

[root@k8s-master mnt]# cat read.yaml
apiVersion: v1
kind: Pod
metadata:
  name: readiness-httpget-pod
  namespace: default
spec:
  containers:
  - name: readiness-httpget-container
    image: wangyanglinux/myapp:v1
    imagePullPolicy: IfNotPresent
    readinessProbe:
      httpGet:
        port: 80
        path: /index1.html
      initialDelaySeconds: 1
      periodSeconds: 3
[root@k8s-master mnt]#
[root@k8s-master mnt]# vim read.yaml
[root@k8s-master mnt]# kubectl create -f read.yaml
pod/readiness-httpget-pod created
[root@k8s-master mnt]# kubectl get pod
NAME                    READY   STATUS    RESTARTS   AGE
myapp-pod               1/1     Running   0          70m
readiness-httpget-pod   0/1     Running   0          17s
[root@k8s-master mnt]# kubectl describe pod readiness-httpget-pod
Name:         readiness-httpget-pod
Namespace:    default
Priority:     0
Node:         k8s-node01/192.168.180.133
Start Time:   Wed, 18 Dec 2019 23:12:59 +0800
Labels:       
Annotations:  
Status:       Running
IP:           10.244.2.10
IPs:
  IP:  10.244.2.10
Containers:
  readiness-httpget-container:
    Container ID:   docker://566ff6cdcf44daaba316b796fb8bf6f9563ddd44000c9ae9f572fd0a6719684c
    Image:          wangyanglinux/myapp:v1
    Image ID:       docker-pullable://wangyanglinux/myapp@sha256:9c3dc30b5219788b2b8a4b065f548b922a34479577befb54b03330999d30d513
    Port:           
    Host Port:      
    State:          Running
      Started:      Wed, 18 Dec 2019 23:13:01 +0800
    Ready:          False
    Restart Count:  0
    Readiness:      http-get http://:80/index1.html delay=1s timeout=1s period=3s #success=1 #failure=3
    Environment:    
    Mounts:
      /var/run/secrets/kubernetes.io/serviceaccount from default-token-gx2h8 (ro)
Conditions:
  Type              Status
  Initialized       True
  Ready             False
  ContainersReady   False
  PodScheduled      True
Volumes:
  default-token-gx2h8:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  default-token-gx2h8
    Optional:    false
QoS Class:       BestEffort
Node-Selectors:  
Tolerations:     node.kubernetes.io/not-ready:NoExecute for 300s
                 node.kubernetes.io/unreachable:NoExecute for 300s
Events:
  Type     Reason     Age                From                 Message
  ----     ------     ----               ----                 -------
  Normal   Scheduled            default-scheduler    Successfully assigned default/readiness-httpget-pod to k8s-node01
  Normal   Pulled     66s                kubelet, k8s-node01  Container image "wangyanglinux/myapp:v1" already present on machine
  Normal   Created    66s                kubelet, k8s-node01  Created container readiness-httpget-container
  Normal   Started    66s                kubelet, k8s-node01  Started container readiness-httpget-container
  Warning  Unhealthy  0s (x22 over 63s)  kubelet, k8s-node01  Readiness probe failed: HTTP probe failed with statuscode: 404
[root@k8s-master mnt]# kubectl exec readiness-httpget-pod -it /bin/sh
/ # ls
bin    dev    etc    home   lib    media  mnt    proc   root   run    sbin   srv    sys    tmp    usr    var
/ # cd /usr/share/nginx
/usr/share/nginx # ls -l
total 0
drwxr-xr-x    1 root     root            24 Feb 25  2018 html
/usr/share/nginx # cd html/
/usr/share/nginx/html # ls -l
total 8
-rw-r--r--    1 root     root           537 Jan 10  2018 50x.html
-rw-r--r--    1 root     root            65 Mar  2  2018 index.html
/usr/share/nginx/html # cat index.html
Hello MyApp | Version: v1 | Pod Name
/usr/share/nginx/html # echo "123" >> index1.html
/usr/share/nginx/html # exit
[root@k8s-master mnt]# kubectl get pod
NAME                    READY   STATUS    RESTARTS   AGE
myapp-pod               1/1     Running   1          73m
readiness-httpget-pod   1/1     Running   0          3m41s

说明:由于index1.html不存在,会导致他重启,手动创建后就正常了。

检测探针 - 存活检测

[root@k8s-master mnt]# cat live-exec.yaml
apiVersion: v1
kind: Pod
metadata:
  name: liveness-exec-pod
  namespace: default
spec:
  containers:
  - name: liveness-exec-container
    image: busybox
    imagePullPolicy: IfNotPresent
    command: ["/bin/sh","-c","touch /tmp/live ; sleep 60; rm -rf /tmp/live; sleep 3600"]
    livenessProbe:
      exec:
        command: ["test","-e","/tmp/live"]
      initialDelaySeconds: 1
      periodSeconds: 3
[root@k8s-master mnt]#
[root@k8s-master mnt]# vim live-exec.yaml
[root@k8s-master mnt]# kubectl create -f live-exec.yaml
pod/liveness-exec-pod created
[root@k8s-master mnt]# kubectl get pod -w
NAME                    READY   STATUS    RESTARTS   AGE
liveness-exec-pod       1/1     Running   0          25s
myapp-pod               1/1     Running   1          81m
readiness-httpget-pod   1/1     Running   0          11m
liveness-exec-pod       1/1     Running   1          101s

liveness-exec-pod       1/1     Running   2          3m19s
^Z
[1]+  已停止               kubectl get pod -w

说明:由于/tmp/live不存在,会一直重启

[root@k8s-master mnt]# cat live-http.yaml
apiVersion: v1
kind: Pod
metadata:
  name: liveness-httpget-pod
  namespace: default
spec:
  containers:
  - name: liveness-httpget-container
    image: wangyanglinux/myapp:v1
    imagePullPolicy: IfNotPresent
    ports:
    - name: http
      containerPort: 80
    livenessProbe:
      httpGet:
        port: http
        path: /index.html
      initialDelaySeconds: 1
      periodSeconds: 3
      timeoutSeconds: 10
[root@k8s-master mnt]#
[root@k8s-master mnt]# kubectl create -f live-http.yaml
pod/liveness-httpget-pod created
[root@k8s-master mnt]# kubectl get pod
NAME                    READY   STATUS    RESTARTS   AGE
liveness-httpget-pod    1/1     Running   0          14s
myapp-pod               1/1     Running   1          90m
readiness-httpget-pod   1/1     Running   0          20m
[root@k8s-master mnt]# kubectl get pod -o wide
NAME                    READY   STATUS    RESTARTS   AGE   IP            NODE         NOMINATED NODE   READINESS GATES
liveness-httpget-pod    1/1     Running   0          26s   10.244.2.12   k8s-node01              
myapp-pod               1/1     Running   1          90m   10.244.1.9    k8s-node02              
readiness-httpget-pod   1/1     Running   0          20m   10.244.2.10   k8s-node01              
[root@k8s-master mnt]# curl 10.244.2.12
Hello MyApp | Version: v1 | Pod Name
[root@k8s-master mnt]# curl 10.244.2.12/index.html
Hello MyApp | Version: v1 | Pod Name
[root@k8s-master mnt]# kubectl exec liveness-httpget-pod -it -- /bin/sh
/ # cd /usr/share/nginx/html/
/usr/share/nginx/html # ls -l
total 8
-rw-r--r--    1 root     root           537 Jan 10  2018 50x.html
-rw-r--r--    1 root     root            65 Mar  2  2018 index.html
/usr/share/nginx/html # rm -rf index.html
/usr/share/nginx/html # exit
[root@k8s-master mnt]# kubectl get pod
NAME                    READY   STATUS    RESTARTS   AGE
liveness-httpget-pod    1/1     Running   0          2m24s
myapp-pod               1/1     Running   1          92m
readiness-httpget-pod   1/1     Running   0          22m
[root@k8s-master mnt]# kubectl get pod
NAME                    READY   STATUS    RESTARTS   AGE
liveness-httpget-pod    1/1     Running   1          2m41s
myapp-pod               1/1     Running   1          92m
readiness-httpget-pod   1/1     Running   0          22m

说明:删除Html,会发现Pod开始重启了。

[root@k8s-master mnt]# cat live-tcp.yaml
apiVersion: v1
kind: Pod
metadata:
  name: probe-tcp
spec:
  containers:
  - name: nginx
    image: wangyanglinux/myapp:v1
    livenessProbe:
      initialDelaySeconds: 5
      timeoutSeconds: 1
      tcpSocket:
        port: 8080
      periodSeconds: 3
[root@k8s-master mnt]#
[root@k8s-master mnt]# vim live-tcp.yaml
[root@k8s-master mnt]# kubectl create -f live-tcp.yaml
pod/probe-tcp created
[root@k8s-master mnt]# kubectl get pod
NAME                    READY   STATUS    RESTARTS   AGE
liveness-httpget-pod    1/1     Running   1          9m24s
myapp-pod               1/1     Running   1          99m
probe-tcp               1/1     Running   0          5s
readiness-httpget-pod   1/1     Running   0          29m
[root@k8s-master mnt]# kubectl get pod
NAME                    READY   STATUS    RESTARTS   AGE
liveness-httpget-pod    1/1     Running   1          9m37s
myapp-pod               1/1     Running   1          99m
probe-tcp               1/1     Running   1          18s
readiness-httpget-pod   1/1     Running   0          29m
[root@k8s-master mnt]# kubectl get pod
NAME                    READY   STATUS    RESTARTS   AGE
liveness-httpget-pod    1/1     Running   1          9m41s
myapp-pod               1/1     Running   1          99m
probe-tcp               1/1     Running   1          22s
readiness-httpget-pod   1/1     Running   0          29m
[root@k8s-master mnt]# kubectl get pod
NAME                    READY   STATUS    RESTARTS   AGE
liveness-httpget-pod    1/1     Running   1          9m43s
myapp-pod               1/1     Running   1          99m
probe-tcp               1/1     Running   1          24s
readiness-httpget-pod   1/1     Running   0          29m
[root@k8s-master mnt]# kubectl get pod
NAME                    READY   STATUS    RESTARTS   AGE
liveness-httpget-pod    1/1     Running   1          9m44s
myapp-pod               1/1     Running   1          99m
probe-tcp               1/1     Running   1          25s
readiness-httpget-pod   1/1     Running   0          29m
[root@k8s-master mnt]# kubectl get pod
NAME                    READY   STATUS    RESTARTS   AGE
liveness-httpget-pod    1/1     Running   1          10m
myapp-pod               1/1     Running   1          100m
probe-tcp               1/1     Running   3          47s
readiness-httpget-pod   1/1     Running   0          29m
[root@k8s-master mnt]# kubectl delete -f live-tcp.yaml
pod "probe-tcp" deleted
[root@k8s-master mnt]#

说明:删除Html,会发现Pod开始重启了。

Pod hook

Pod hook (钩子)是由 Kubernetes 管理的 kubelet 发起的,当容器中的进程启动前或者容器中的进
程终止之前运行,这是包含在容器的生命周期之中。可以同时为 Pod 中的所有容器都配置 hook
Hook 的类型包括两种:
exec :执行一段命令
HTTP :发送 HTTP 请求

重启策略

PodSpec 中有一个 restartPolicy 字段,可能的值为 Always 、 OnFailure 和 Never 。默认为
Always 。 restartPolicy 适用于 Pod 中的所有容器。 restartPolicy 仅指通过同一节点上的
kubelet 重新启动容器。失败的容器由 kubelet 以五分钟为上限的指数退避延迟( 10 秒, 20 秒, 40
秒 ... )重新启动,并在成功执行十分钟后重置。如 Pod 文档 中所述,一旦绑定到一个节点, Pod 将
永远不会重新绑定到另一个节点。

Pod phase

Pod 的 status 字段是一个 PodStatus 对象, PodStatus 中有一个 phase 字段。
Pod 的相位( phase )是 Pod 在其生命周期中的简单宏观概述。该阶段并不是对容器或 Pod 的综合汇总,也不是为了做为综合状态机
Pod 相位的数量和含义是严格指定的。除了本文档中列举的状态外,不应该再假定 Pod 有其他的phase 值
几种常见的值

  • 挂起( Pending ): Pod 已被 Kubernetes 系统接受,但有一个或者多个容器镜像尚未创建。等待时间包括调度 Pod 的时间和通过网络下载镜像的时间,这可能需要花点时间
  • 运行中( Running ):该 Pod 已经绑定到了一个节点上, Pod 中所有的容器都已被创建。至少有一个容器正在运行,或者正处于启动或重启状态
  • 成功( Succeeded ): Pod 中的所有容器都被成功终止,并且不会再重启
  • 失败( Failed ): Pod 中的所有容器都已终止了,并且至少有一个容器是因为失败终止。也就是说,容器以非 0 状态退出或者被系统终止
  • 未知( Unknown ):因为某些原因无法取得 Pod 的状态,通常是因为与 Pod 所在主机通信失败
[root@k8s-master mnt]# vim post.yaml
[root@k8s-master mnt]# kubectl create -f post.yaml
pod/lifecycle-demo created
[root@k8s-master mnt]# kubectl get pod
NAME                    READY   STATUS    RESTARTS   AGE
lifecycle-demo          1/1     Running   0          9s
liveness-httpget-pod    1/1     Running   1          40m
myapp-pod               1/1     Running   1          130m
readiness-httpget-pod   1/1     Running   0          60m
[root@k8s-master mnt]# kubectl exec lifecycle-demo -it -- /bin/bash
OCI runtime exec failed: exec failed: container_linux.go:346: starting container process caused "exec: \"/bin/bash\": stat /bin/bash: no such file or directory": unknown
command terminated with exit code 126
[root@k8s-master mnt]# kubectl exec lifecycle-demo -it -- /bin/sh
/ # cd /usr/share/message
/bin/sh: cd: can't cd to /usr/share/message
/ # cat /usr/share/message
Hello from the postStart handler
/ # exit
[root@k8s-master mnt]# cat post.yaml
apiVersion: v1
kind: Pod
metadata:
  name: lifecycle-demo
spec:
  containers:
  - name: lifecycle-demo-container
    image: wangyanglinux/myapp:v1
    lifecycle:
      postStart:
        exec:
          command: ["/bin/sh", "-c", "echo Hello from the postStart handler > /usr/share/message"]
      preStop:
        exec:
          command: ["/bin/sh", "-c", "echo Hello from the poststop handler > /usr/share/message"]
[root@k8s-master mnt]#

原文链接:http://dwz.date/MMr