K8s弹性伸缩-HPA（Horizontal Pod Autoscaler ）

简介

Pod 水平自动扩缩（Horizontal Pod Autoscaler）可以基于 CPU 利用率自动扩缩 ReplicationController、Deployment、ReplicaSet 和 StatefulSet 中的 Pod 数量。除了 CPU 利用率，也可以基于其他应程序提供的自定义度量指标来执行自动扩缩。 Pod 自动扩缩不适用于无法扩缩的对象，比如 DaemonSet。

Pod 水平自动扩缩特性由 Kubernetes API 资源和控制器实现。资源决定了控制器的行为。控制器会周期性的调整副本控制器或 Deployment 中的副本数量，以使得 Pod 的平均 CPU 利用率与用户所设定的目标值匹配。

Pod自动扩缩机制

image.png

Pod 水平自动扩缩器的实现是一个控制回路，由控制器管理器的 --horizontal-pod-autoscaler-sync-period 参数指定周期（默认值为 15 秒）。

每个周期内，控制器管理器根据每个 HorizontalPodAutoscaler 定义中指定的指标查询资源利用率。控制器管理器可以从资源度量指标 API（按 Pod 统计的资源用量）和自定义度量指标 API（其他指标）获取度量值。

API对象

HorizontalPodAutoscaler 是 Kubernetes autoscaling API 组的资源。在当前稳定版本（autoscaling/v1）中只支持基于 CPU 指标的扩缩。

API 的 beta 版本（autoscaling/v2beta2）引入了基于内存和自定义指标的扩缩。在 autoscaling/v2beta2 版本中新引入的字段在 autoscaling/v1 版本中以注解的形式得以保留。

kubectl对HPA支持

与其他 API 资源类似，kubectl 以标准方式支持 HPA。我们可以通过 kubectl create 命令创建一个 HPA 对象，通过 kubectl get hpa 命令来获取所有 HPA 对象，通过 kubectl describe hpa 命令来查看 HPA 对象的详细信息。最后，可以使用 kubectl delete hpa 命令删除对象。

此外，还有个简便的命令 kubectl autoscale 来创建 HPA 对象。例如，命令 kubectl autoscale rs foo --min=2 --max=5 --cpu-percent=80 将会为名为 foo 的 ReplicationSet 创建一个 HPA 对象，目标 CPU 使用率为 80%，副本数量配置为 2 到 5 之间。

滚动升级收缩

目前在 Kubernetes 中，可以针对 ReplicationController 或 Deployment 执行滚动更新，它们会为你管理底层副本数。 Pod 水平扩缩只支持后一种：HPA 会被绑定到 Deployment 对象， HPA 设置副本数量时，Deployment 会设置底层副本数。

通过直接操控副本控制器执行滚动升级时，HPA 不能工作，也就是说你不能将 HPA 绑定到某个 RC 再执行滚动升级。 HPA 不能工作的原因是它无法绑定到滚动更新时所新创建的副本控制器。
Horizontal Pod Autoscaler 可以根据 CPU 利用率自动扩缩 ReplicationController、 Deployment、ReplicaSet 或 StatefulSet 中的 Pod 数量（也可以基于其他应用程序提供的度量指标，目前这一功能处于 beta 版本）。

部署一个HPA

提前部署好一个metrics-server，以便通过 Metrics API 提供度量数据。 Horizontal Pod Autoscaler 根据此 API 来获取度量数据

创建一个deployment并暴露服务

apiVersion: apps/v1
kind: Deployment
metadata:
  name: myapp
  namespace: default
spec:
  replicas: 1
  selector:
    matchLabels:
      app: myapp
  template:
    metadata:
      labels:
        app: myapp
    spec:
      containers:
      - name: myapp
        image: ikubernetes/myapp:v1
        imagePullPolicy: IfNotPresent
        ports: 
        - name: http
          containerPort: 80
        resources:
          requests:
            memory: "50Mi"
            cpu: "200m"
          limits:
            memory: "50Mi"     
            cpu: "200m"
---
apiVersion: v1
kind: Service
metadata:
  name: myapp
  labels:
    app: myapp
spec:
  type: NodePort
  ports:
  - port: 80
    targetPort: 80
  selector:
    app: myapp


[root@k8s-master ~]# kubectl get pods 
NAME                     READY   STATUS    RESTARTS   AGE
myapp-6994cb56cb-5k5qv   1/1     Running   0          4m14s
[root@k8s-master ~]# kubectl get svc 
NAME         TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)   AGE
kubernetes   ClusterIP   10.96.0.1               443/TCP   4d6h
myapp        ClusterIP   10.96.124.235           80/TCP    4m18s

创建一个HPA

[root@k8s-master ~]# kubectl autoscale deployment myapp --min=1 --max=8 --cpu-percent=60 
horizontalpodautoscaler.autoscaling/myapp autoscaled
[root@k8s-master ~]# kubectl get hpa
NAME    REFERENCE          TARGETS         MINPODS   MAXPODS   REPLICAS   AGE
myapp   Deployment/myapp   /60%   1         8         0          8s
[root@k8s-master ~]# kubectl describe hpa myapp
Name:                                                  myapp
Namespace:                                             default
Labels:                                                
Annotations:                                           
CreationTimestamp:                                     Tue, 20 Apr 2021 17:07:24 +0800
Reference:                                             Deployment/myapp
Metrics:                                               ( current / target )
  resource cpu on pods  (as a percentage of request):  0% (0) / 60%
Min replicas:                                          1
Max replicas:                                          5
Deployment pods:                                       1 current / 1 desired
Conditions:
  Type            Status  Reason               Message
  ----            ------  ------               -------
  AbleToScale     True    ScaleDownStabilized  recent recommendations were higher than current one, applying the highest recent recommendation
  ScalingActive   True    ValidMetricFound     the HPA was able to successfully calculate a replica count from cpu resource utilization (percentage of request)
  ScalingLimited  False   DesiredWithinRange   the desired count is within the acceptable range
Events:

进行压测增加负载

[root@k8s-node1 ~]# ab -c 500 -n  5000000 http://10.0.0.11:32644/index.html

#查看hpa
[root@k8s-master ~]# kubectl describe hpa myapp
Name:                                                  myapp
Namespace:                                             default
Labels:                                                
Annotations:                                           
CreationTimestamp:                                     Tue, 20 Apr 2021 17:07:24 +0800
Reference:                                             Deployment/myapp
Metrics:                                               ( current / target )
  resource cpu on pods  (as a percentage of request):  91% (182m) / 60%
Min replicas:                                          1
Max replicas:                                          8
Deployment pods:                                       4 current / 4 desired
Conditions:
  Type            Status  Reason              Message
  ----            ------  ------              -------
  AbleToScale     True    ReadyForNewScale    recommended size matches current size
  ScalingActive   True    ValidMetricFound    the HPA was able to successfully calculate a replica count from cpu resource utilization (percentage of request)
  ScalingLimited  False   DesiredWithinRange  the desired count is within the acceptable range
Events:
  Type    Reason             Age                 From                       Message
  ----    ------             ----                ----                       -------
  Normal  SuccessfulRescale  16m                 horizontal-pod-autoscaler  New size: 3; reason: cpu resource utilization (percentage of request) above target
  Normal  SuccessfulRescale  15m                 horizontal-pod-autoscaler  New size: 5; reason: cpu resource utilization (percentage of request) above target
  Normal  SuccessfulRescale  2m59s               horizontal-pod-autoscaler  New size: 1; reason: All metrics below target
  Normal  SuccessfulRescale  103s (x2 over 18m)  horizontal-pod-autoscaler  New size: 2; reason: cpu resource utilization (percentage of request) above target
  Normal  SuccessfulRescale  43s                 horizontal-pod-autoscaler  New size: 4; reason: cpu resource utilization (percentage of request) above target
[root@k8s-master ~]# kubectl get pods 
NAME                     READY   STATUS    RESTARTS   AGE
myapp-6994cb56cb-5k5qv   1/1     Running   0          45m
myapp-6994cb56cb-cvfxb   1/1     Running   0          105s
myapp-6994cb56cb-knzbd   1/1     Running   0          45s
myapp-6994cb56cb-zkp79   1/1     Running   0          45s

#使用kubectl命令默认创建的是autoscaling/v1控制器，v1控制器只能基于cpu负载进行自动扩缩容，而v2支持多维度的值自动扩缩容

使用autoscaling/v2beta1控制器创建

apiVersion: autoscaling/v2beta1
kind: HorizontalPodAutoscaler
metadata:
  name: myapp-hpa-v2
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: myapp 
  minReplicas: 1
  maxReplicas: 10
  metrics: 
  - type: Resource
    resource: 
      name: cpu
      targetAverageUtilization: 55
  - type: Resource
    resource: 
      name: memory
      targetAverageValue: 50Mi

查看

[root@k8s-master ~]# kubectl apply -f hpa.yaml 
horizontalpodautoscaler.autoscaling/myapp-hpa-v2 created
[root@k8s-master ~]# kubectl get hpa
NAME           REFERENCE          TARGETS                         MINPODS   MAXPODS   REPLICAS   AGE
myapp-hpa-v2   Deployment/myapp   /50Mi, /55%   1         10        0          7s

压测

[root@k8s-node1 ~]# ab -c 500 -n  5000000 http://10.0.0.11:32644/index.html

[root@k8s-master ~]# kubectl describe hpa myapp
Name:                                                  myapp-hpa-v2
Namespace:                                             default
Labels:                                                
Annotations:                                           CreationTimestamp:  Tue, 20 Apr 2021 17:59:02 +0800
Reference:                                             Deployment/myapp
Metrics:                                               ( current / target )
  resource memory on pods:                             3432448 / 50Mi
  resource cpu on pods  (as a percentage of request):  89% (179m) / 55%
Min replicas:                                          1
Max replicas:                                          10
Deployment pods:                                       4 current / 4 desired
Conditions:
  Type            Status  Reason              Message
  ----            ------  ------              -------
  AbleToScale     True    ReadyForNewScale    recommended size matches current size
  ScalingActive   True    ValidMetricFound    the HPA was able to successfully calculate a replica count from cpu resource utilization (percentage of request)
  ScalingLimited  False   DesiredWithinRange  the desired count is within the acceptable range
Events:
  Type    Reason             Age   From                       Message
  ----    ------             ----  ----                       -------
  Normal  SuccessfulRescale  79s   horizontal-pod-autoscaler  New size: 2; reason: cpu resource utilization (percentage of request) above target
  Normal  SuccessfulRescale  18s   horizontal-pod-autoscaler  New size: 4; reason: cpu resource utilization (percentage of request) above target

[root@k8s-master ~]# kubectl get pods 
NAME                     READY   STATUS    RESTARTS   AGE
myapp-6994cb56cb-25cs6   1/1     Running   0          32s
myapp-6994cb56cb-5k5qv   1/1     Running   0          70m
myapp-6994cb56cb-t4ksp   1/1     Running   0          32s
myapp-6994cb56cb-z42bc   1/1     Running   0          93s
#注：停止压测后会自动伸缩