十九、k8s如何优先删除利用率低的Pod

        同一应用的不通Pod可能其利用率是不同的。在对应用执行缩容时，希望移除利用率较低 的Pod.

        通过使用 controller.kubernetes.io/pod-deletion-cost 注解，用户可以对 ReplicaSet 缩容时要先删除哪些 Pod 设置偏好。

• 类别：注解
• 特性状态： Kubernetes v1.22 [beta]
• 例子：controller.kubernetes.io/pod-deletion-cost: "10"
• 用于：Pod
• 该注解用于设置Pod 删除成本允许用户影响 ReplicaSet 缩减顺序。注解解析为 int32 类型。

示例：

        编写一个副本数为3的nginx-deployment

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-deployment
spec:
  selector:
    matchLabels:
      app: nginx

  replicas: 3 
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: nginx:1.14.2
        ports:
        - containerPort: 80

部署后 效果如下：

$ kubectl apply -f nginx-deployment.yaml 
deployment.apps/nginx-deployment created

$  kubectl get pod -owide
NAME                                READY   STATUS    RESTARTS   AGE   IP              NODE     NOMINATED NODE   READINESS GATES
nginx-deployment-66b6c48dd5-97rjj   1/1     Running   0          11s   10.244.219.78   master              
nginx-deployment-66b6c48dd5-fcckz   1/1     Running   0          11s   10.244.219.77   master              
nginx-deployment-66b6c48dd5-rsq8j   1/1     Running   0          11s   10.244.219.79   master              

        修改pod注解，使用 controller.kubernetes.io/pod-deletion-cost 注解

nginx-deployment-66b6c48dd5-97rjj 删除成本设置为100
nginx-deployment-66b6c48dd5-fcckz删除成本设置为60
nginx-deployment-66b6c48dd5-rsq8j删除成本设置为20
`nginx-deployment-66b6c48dd5-97rjj `删除成本设置为100
kubectl annotate pod nginx-deployment-66b6c48dd5-97rjj controller.kubernetes.io/pod-deletion-cost=100 --overwrite

# `nginx-deployment-66b6c48dd5-fcckz`删除成本设置为60
kubectl annotate pod nginx-deployment-66b6c48dd5-fcckz controller.kubernetes.io/pod-deletion-cost=60 --overwrite

# `nginx-deployment-66b6c48dd5-rsq8j`删除成本设置为20
kubectl annotate pod nginx-deployment-66b6c48dd5-rsq8j controller.kubernetes.io/pod-deletion-cost=20 --overwrite

        查看所有pod的controller.kubernetes.io/pod-deletion-cost注解，发现 controller.kubernetes.io/pod-deletion-cost 注解已经被添加

$ kubectl get pods -l app=nginx -o custom-columns=POD_NAME:.metadata.name,POD_DELETION_COST:.metadata.annotations.'controller\.kubernetes\.io/pod-deletion-cost'
POD_NAME                            POD_DELETION_COST
nginx-deployment-66b6c48dd5-97rjj   100
nginx-deployment-66b6c48dd5-fcckz   60
nginx-deployment-66b6c48dd5-rsq8j   20

        使用kubectl scale命令缩容nginx-deployment，发现controller.kubernetes.io/pod-deletion-cost数值低的pod优先被删除

$ kubectl scale deployment nginx-deployment --replicas=2
deployment.apps/nginx-deployment scaled

# 发现`controller.kubernetes.io/pod-deletion-cost`数值低的pod优先被删除
$  kubectl get pod -owide
NAME                                READY   STATUS    RESTARTS   AGE   IP              NODE     NOMINATED NODE   READINESS GATES
nginx-deployment-66b6c48dd5-97rjj   1/1     Running   0          12m   10.244.219.78   master              
nginx-deployment-66b6c48dd5-fcckz   1/1     Running   0          12m   10.244.219.77   master              

1.  如果pod没分配到节点，先被删除
2. 如果pod的状态是 Pending>PodUnknown>PodRunning，则Pending优先被删除，PodUnknown次之，PodRunning最后被删除。
3. 不是Ready状态的Pod先被删除。
4. 如果Pod都是Ready状态，则最后一个变成Ready状态的Pod先被删除（Ready时间最短的）。
5. 重启次数大的Pod先被删除。
6. 创建时间最新的Pod先被删除。