kubernetes之Deployments

Deployments是kubernetes中的一种控制器，是比ReplicaSet更高级的概念，它最重的特性是支持对pod与ReplicaSet的声明式升级，声明式升级比其它方式的升级更安全可靠。需要注意的是用户不应该手动管理被Deployments创建的ReplicaSet。

使用案例

以下是几种典型的Deployments使用案例：

• Create a Deployment to rollout a ReplicaSet. The ReplicaSet creates Pods in the background. Check the status of the rollout to see if it succeeds or not.
• Declare the new state of the Pods by updating the PodTemplateSpec of the Deployment. A new ReplicaSet is created and the Deployment manages moving the Pods from the old ReplicaSet to the new one at a controlled rate. Each new ReplicaSet updates the revision of the Deployment.
• Rollback to an earlier Deployment revision if the current state of the Deployment is not stable. Each rollback updates the revision of the Deployment.
• Scale up the Deployment to facilitate more load.
• Pause the Deployment to apply multiple fixes to its PodTemplateSpec and then resume it to start a new rollout.
• Use the status of the Deployment as an indicator that a rollout has stuck.
• Clean up older ReplicaSets that you don’t need anymore

Creating a Deployment

以下Deployment示例创建一个ReplicaSet，ReplicaSet控制三个nginx pod，示下是配置文件：

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-deployment
  labels:
    app: nginx
spec:
  replicas: 3
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: nginx:1.7.9
        ports:
        - containerPort: 80

以上文件中除了kind变成Deployment外，与ReplicaSet的定义并无不同，Deployment据此创建ReplicaSet。

创建Deployment： 

kubectl create -f  https://k8s.io/examples/controllers/nginx-deployment.yaml

提示：可以在上述命令中追加--record选项，其作用是在Deployment的注解中记录下当前执行的命令，这个特性对于Deployment的变更审计、追踪很有用。

 查看Deployments：

NAME               DESIRED   CURRENT   UP-TO-DATE   AVAILABLE   AGE
nginx-deployment   3         0         0            0           1s

输出的自字段含义：

• NAME： Deployments名称
• DESIRED：创建Deployments时指定的期望副本数量。
• CURRENT：当前正在运行的副本数量。
• UP-TO-DATE：成功升级的副本数量。
• AVAILABLE：可用副本的数量，注意副本在运行不等于可用。
• AGE：应用运行时间。 

创建Deployments后，kubernetes在系统中保存代表Deployments规格与状态的数据，以上字段与系统中保存数据的对应关系：

• DESIRED：spec.replicas field.
• CURRENT：status.replicas field.
• UP-TO-DATE：status.updatedReplicas field.
• AVAILABLE：status.availableReplicas field.

查看Deployments的推进状态(rollout status)：

kubectl rollout status deployment/nginx-deployment
Waiting for rollout to finish: 2 out of 3 new replicas have been updated...
deployment "nginx-deployment" successfully rolled out

结果显示正在等待部署完成，3个副本中的2两已经更新完成，稍后查看Deployments的状态：

kubectl get deployments
NAME               DESIRED   CURRENT   UP-TO-DATE   AVAILABLE   AGE
nginx-deployment   3         3         3            3           18s

 以上结果表明，3个副本创建完成，全部可用，UP-TO-DATE表示其状态与Deployment最新的配置匹配。

查看Deployments创建的ReplicaSet：

kubectl get rs
NAME                          DESIRED   CURRENT   READY   AGE
nginx-deployment-2035384211   3         3         3       18s

注意“NAME” 列值，其格式为[DEPLOYMENT-NAME]-[POD-TEMPLATE-HASH-VALUE]，哈希值在创建时自动生成。注意这个HASH值非常重要，在后边的例子中可以看到，对Deployment进行更新时，它会创建新的ReplicaSet代替旧ReplicaSet，此时一个Deployment有新旧两个ReplicaSet，区别它们的方式就是这个HASH值。那么有两个ReplicaSet，在更新的过程就会存在新旧两个版本的pod，如何区分新旧版pod不至于引起管理权的混淆？答案也是这个HASH值，系统自动将这个值添加到pod的标签中，这样旧ReplicaSet 与旧pod对应，新ReplicaSet与新pod对应，运行如下命令确认：

kubectl get pods --show-labels
NAME                                READY     STATUS    RESTARTS   AGE       LABELS
nginx-deployment-2035384211-7ci7o   1/1       Running   0          18s       app=nginx,pod-template-hash=2035384211
nginx-deployment-2035384211-kzszj   1/1       Running   0          18s       app=nginx,pod-template-hash=2035384211
nginx-deployment-2035384211-qqcnn   1/1       Running   0          18s       app=nginx,pod-template-hash=2035384211

注意其中的pod-template-hash。

Updating a Deployment

重要提示：只有Deployment的.spec.template部分的内容变更时才会触发rollout，rollout是更新，它涉及增与减两个操作，相对复杂。其它变更比如修改.spec.replicas的值，不会触发rollout，因为它只是单纯的增或者减，相对简单。

假设将nginx:1.7.9更新到nginx:1.9.1：

$ kubectl set image deployment/nginx-deployment nginx=nginx:1.9.1
deployment "nginx-deployment" image updated

 其它方法，通过edit命令编辑实时对象：

$ kubectl edit deployment/nginx-deployment
deployment "nginx-deployment" edited

 在编辑器中将.spec.template.spec.containers[0].image的值从nginx:1.7.9改成nginx:1.9.1。

查看rollout状态，也就是查看更新进度：

$ kubectl rollout status deployment/nginx-deployment
Waiting for rollout to finish: 2 out of 3 new replicas have been updated...
deployment "nginx-deployment" successfully rolled out

当显示rollout成功后，查看Deployments状态：

$ kubectl get deployments
NAME               DESIRED   CURRENT   UP-TO-DATE   AVAILABLE   AGE
nginx-deployment   3         3         3            3           36s

由以上内容可知道更新过程已经成功结束。DESIRED表示预期pod数量是3，CRRENT表示当前数量也是3，注意在rollout的过程中，如果新的pod正在创建但旧的pod还没有销毁，那么这个字段的数值就会大小3。UP-TO-DATE表示新pod的个数，现在与DESIRED的值相等都是3，表示已经全部更新到新的pod，在rollout的过程中这个值是逐渐增大的。AVAILABLE是3，表示目前新的pod都可用，但是在rollout的过程中此值有可能会小于DESIRED与CURRENT的值，原因就是新pod还不可用而旧pod已经开始删除。 

Deployment在更新时会创建新的ReplicaSet，然后逐渐放大新ReplicaSet的副本数量一直到预期数量。同时逐渐缩小旧ReplicaSet的副本数量一直到0，运行如下命令确认，可以看到现在系统中有新旧两个ReplicaSet，需要注意旧的ReplicaSet仍然保存在系统中，应该是到期以后会被当成垃圾回收：

$ kubectl get rs
NAME                          DESIRED   CURRENT   READY   AGE
nginx-deployment-1564180365   3         3         3       6s
nginx-deployment-2035384211   0         0         0       36s

 get pods命令显示的是新创建的pod，旧pod已经被删除：

$ kubectl get pods
NAME                                READY     STATUS    RESTARTS   AGE
nginx-deployment-1564180365-khku8   1/1       Running   0          14s
nginx-deployment-1564180365-nacti   1/1       Running   0          14s
nginx-deployment-1564180365-z9gth   1/1       Running   0          14s

获取Deployments描述：

$ kubectl describe deployments
Name:                   nginx-deployment
Namespace:              default
CreationTimestamp:      Thu, 30 Nov 2017 10:56:25 +0000
Labels:                 app=nginx
Annotations:            deployment.kubernetes.io/revision=2
Selector:               app=nginx
Replicas:               3 desired | 3 updated | 3 total | 3 available | 0 unavailable
StrategyType:           RollingUpdate
MinReadySeconds:        0
RollingUpdateStrategy:  25% max unavailable, 25% max surge
Pod Template:
  Labels:  app=nginx
  Containers:
   nginx:
    Image:        nginx:1.9.1
    Port:         80/TCP
    Environment:  
    Mounts:       
  Volumes:        
Conditions:
  Type           Status  Reason
  ----           ------  ------
  Available      True    MinimumReplicasAvailable
  Progressing    True    NewReplicaSetAvailable
OldReplicaSets:  
NewReplicaSet:   nginx-deployment-1564180365 (3/3 replicas created)
Events:
  Type    Reason             Age   From                   Message
  ----    ------             ----  ----                   -------
  Normal  ScalingReplicaSet  2m    deployment-controller  Scaled up replica set nginx-deployment-2035384211 to 3
  Normal  ScalingReplicaSet  24s   deployment-controller  Scaled up replica set nginx-deployment-1564180365 to 1
  Normal  ScalingReplicaSet  22s   deployment-controller  Scaled down replica set nginx-deployment-2035384211 to 2
  Normal  ScalingReplicaSet  22s   deployment-controller  Scaled up replica set nginx-deployment-1564180365 to 2
  Normal  ScalingReplicaSet  19s   deployment-controller  Scaled down replica set nginx-deployment-2035384211 to 1
  Normal  ScalingReplicaSet  19s   deployment-controller  Scaled up replica set nginx-deployment-1564180365 to 3
  Normal  ScalingReplicaSet  14s   deployment-controller  Scaled down replica set nginx-deployment-2035384211 to 0

 在上述输出中，可以看到自动添加到注解中的“deployment.kubernetes.io/revision=2”字段，新建Deployment的revision是1，2表示这是更新过的第二版，前缀“deployment.kubernetes.io”表示这是系统保留的供deployment使用的关键字，用户不可以使用引种类型的注解。

"RollingUpdateStrategy:  25% max unavailable, 25% max surge"是默认流动升级策略，前边的%25表示在升级过种不可用副本相对于期望值的比例。以上例计算3*25%＝0.75，省略掉小数后是0，那么在升级的过程中必需有3个pod可用，结果就是必需先创建一个新的pod并等待它可用后才可以删除一个旧的pod。后边的25%表示激增率，结果是0.75取整是1，结果就是次都只能创建1个新的pod。总之这两个值的目的有两个，前一个是在更新时确保有足够可用的pod，后一个限制新旧pod的总数不能超过一定数量。

输出中的Events部分可以看到详细的更新过程，可以看到它是先建一个新pod，然后删除一个旧pod，如此往复直到最后完成。

存在着多重更新的情况，比如上一次更新还没有结束，rollout正在进行中，又进行了一次更新，此时Deployment不会等待上一次更新完成再执行这一次的更新，它在现在的基础上立刻展开本次更新。

上例中的更新，Deployment的.spec.template内容发生变化，从而触发rollout执行更新过程。.spec.template之外的部分，比如标签选择器发生变更，Deployment所采取的动作与触发rollout不同，它是另一个过程。Kubernetes官方不推荐更新标签选择器，它希望标签选择器一开始就被确定下来，后续无需修改。如果非要修改，一定要谨慎、考虑周全。

关于更新失败

如果在新版本中指定了错误的image名称，那么更新注定会失败。在本例中如果发生这种情况，首先Deployment会创建新的ReplicaSet，新ReplicaSet在创建每个pod实例就会因为取不到image而失败，整个更新过程会被卡住不再进行下去。即使失败也会产生一个新的修订版本号，并且新创建的ReplicaSet、因无法取到image而失败的pod仍存在于系统中。当然目前对外提供服务的仍然是旧版本的pod。如果发生这种情况就要乃至回滚功能，回到上一个正确的版本。

Rolling Back a Deployment

Deployment会记录修订历史，当然对能记录的修订历史个数有限制，可以通过设置修改，因此Deployment能回滚到记录在案的以前的历史版本中。注意只有rollout操作才会触发修订历史记录，也就是只有.spec.template发生变更才被认为产生了新的版本。其它的变更，比如修改标签选择器、通过kubectl scalling命令扩缩容、被HPA自动扩缩容等不会被认为是新的版本。只回滚.spec.template部分，比如当前的副本数量是5，回滚的版本副本数量是3，回滚后副本数量不变仍然是5，但副本的内容变回以前。

Kubernetes官方文档提供的回滚示例用的是旧版本，与当前的新版本有所冲突，命令一样，输出有小的差别。以下展示回滚需要用到的命令。

查看修定历史记录，注意后边的CHANGE-CAUSE，需要在kubectl creae创建Deployment时指定--record才会有：

$ kubectl rollout history deployment/nginx-deployment
deployments "nginx-deployment"
REVISION    CHANGE-CAUSE
1           kubectl create -f https://k8s.io/examples/controllers/nginx-deployment.yaml --record
2           kubectl set image deployment/nginx-deployment nginx=nginx:1.9.1
3           kubectl set image deployment/nginx-deployment nginx=nginx:1.91

通过指定REVISION号码查看特定修订记录：

$ kubectl rollout history deployment/nginx-deployment --revision=2
deployments "nginx-deployment" revision 2
  Labels:       app=nginx
          pod-template-hash=1159050644
  Annotations:  kubernetes.io/change-cause=kubectl set image deployment/nginx-deployment nginx=nginx:1.9.1
  Containers:
   nginx:
    Image:      nginx:1.9.1
    Port:       80/TCP
     QoS Tier:
        cpu:      BestEffort
        memory:   BestEffort
    Environment Variables:      
  No volumes.

 回滚到前一个版本：

$ kubectl rollout undo deployment/nginx-deployment
deployment "nginx-deployment" rolled back

通过revision号回滚到任意版本：

$ kubectl rollout undo deployment/nginx-deployment --to-revision=2
deployment "nginx-deployment" rolled back

 查看Deployment描述：

$ kubectl get deployment
NAME               DESIRED   CURRENT   UP-TO-DATE   AVAILABLE   AGE
nginx-deployment   3         3         3            3           30m

$ kubectl describe deployment
Name:           nginx-deployment
Namespace:      default
CreationTimestamp:  Tue, 15 Mar 2016 14:48:04 -0700
Labels:         app=nginx
Selector:       app=nginx
Replicas:       3 updated | 3 total | 3 available | 0 unavailable
StrategyType:       RollingUpdate
MinReadySeconds:    0
RollingUpdateStrategy:  1 max unavailable, 1 max surge
OldReplicaSets:     
NewReplicaSet:      nginx-deployment-1564180365 (3/3 replicas created)
Events:
  FirstSeen LastSeen    Count   From                    SubobjectPath   Type        Reason              Message
  --------- --------    -----   ----                    -------------   --------    ------              -------
  30m       30m         1       {deployment-controller }                Normal      ScalingReplicaSet   Scaled up replica set nginx-deployment-2035384211 to 3
  29m       29m         1       {deployment-controller }                Normal      ScalingReplicaSet   Scaled up replica set nginx-deployment-1564180365 to 1
  29m       29m         1       {deployment-controller }                Normal      ScalingReplicaSet   Scaled down replica set nginx-deployment-2035384211 to 2
  29m       29m         1       {deployment-controller }                Normal      ScalingReplicaSet   Scaled up replica set nginx-deployment-1564180365 to 2
  29m       29m         1       {deployment-controller }                Normal      ScalingReplicaSet   Scaled down replica set nginx-deployment-2035384211 to 0
  29m       29m         1       {deployment-controller }                Normal      ScalingReplicaSet   Scaled up replica set nginx-deployment-3066724191 to 2
  29m       29m         1       {deployment-controller }                Normal      ScalingReplicaSet   Scaled up replica set nginx-deployment-3066724191 to 1
  29m       29m         1       {deployment-controller }                Normal      ScalingReplicaSet   Scaled down replica set nginx-deployment-1564180365 to 2
  2m        2m          1       {deployment-controller }                Normal      ScalingReplicaSet   Scaled down replica set nginx-deployment-3066724191 to 0
  2m        2m          1       {deployment-controller }                Normal      DeploymentRollback  Rolled back deployment "nginx-deployment" to revision 2
  29m       2m          2       {deployment-controller }                Normal      ScalingReplicaSet   Scaled up replica set nginx-deployment-1564180365 to 3

可以看到在Deployment的Events中会记录回滚事件。

Scaling a Deployment

可以是如下命令：

$ kubectl scale deployment nginx-deployment --replicas=10
deployment "nginx-deployment" scaled

HPA方式：

$ kubectl autoscale deployment nginx-deployment --min=10 --max=15 --cpu-percent=80
deployment "nginx-deployment" autoscaled

扩缩容存在一种特殊情况，假如有如下的Deployment：

$ kubectl get deploy
NAME                 DESIRED   CURRENT   UP-TO-DATE   AVAILABLE   AGE
nginx-deployment     10        10        10           10          50s

然后更新image：

$ kubectl set image deploy/nginx-deployment nginx=nginx:sometag
deployment "nginx-deployment" image updated

会出现如下局面：

$ kubectl get rs
NAME                          DESIRED   CURRENT   READY     AGE
nginx-deployment-1989198191   5         5         0         9s
nginx-deployment-618515232    8         8         8         1m

前边说过，在更新的过程中要保证可用pod的数量(8)，同时新旧pod的总数又不能超过一定的限制(13)，只有新版本的5个pod，其中有变成就绪状态后，更新才会继续，现在卡住了。如果此时调用了kubectl scalling命令将期望值由10调整成15，那么新多出的5个pod分配给谁呢？全部分配给新版本不可行，因为新版本迟迟无法就绪，也许永远都无法就绪。如果分配给旧版本的话也不行，因为旧版本正在从更新。Kubernetes采用的策略是按比例分配的折中处理方法，这种方法无论是对于更新成功，还是更新失败而回滚都是可以接受的。如下：

$ kubectl get deploy
NAME                 DESIRED   CURRENT   UP-TO-DATE   AVAILABLE   AGE
nginx-deployment     15        18        7            8           7m
$ kubectl get rs
NAME                          DESIRED   CURRENT   READY     AGE
nginx-deployment-1989198191   7         7         0         7m
nginx-deployment-618515232    11        11        11        7m

 Pausing and Resuming a Deployment

注意这里暂停与唤醒的对象是Deployment，不是pod。暂停以后即使更新Deployment也不会触发rollout。因此可以无暂停Deployment，然后对它进行多次更新，再唤醒，然后rollout就会将多次更新一次性的应用到pod中。

例如有如下Deploymnet：

$ kubectl get deploy
NAME      DESIRED   CURRENT   UP-TO-DATE   AVAILABLE   AGE
nginx     3         3         3            3           1m
$ kubectl get rs
NAME               DESIRED   CURRENT   READY     AGE
nginx-2142116321   3         3         3         1m

运行如下命令暂停：

$ kubectl rollout pause deployment/nginx-deployment
deployment "nginx-deployment" paused

 更新image：

$ kubectl set image deploy/nginx-deployment nginx=nginx:1.9.1
deployment "nginx-deployment" image updated

查看rollout状态，可以看到其并没有触发：

$ kubectl rollout history deploy/nginx-deployment
deployments "nginx"
REVISION  CHANGE-CAUSE
1   

$ kubectl get rs
NAME               DESIRED   CURRENT   READY     AGE
nginx-2142116321   3         3         3         2m

 对Deployment执行更多变更：

$ kubectl set resources deployment nginx-deployment -c=nginx --limits=cpu=200m,memory=512Mi
deployment "nginx-deployment" resource requirements updated

唤醒并查看 rollout：

$ kubectl rollout resume deploy/nginx-deployment
deployment "nginx" resumed
$ kubectl get rs -w
NAME               DESIRED   CURRENT   READY     AGE
nginx-2142116321   2         2         2         2m
nginx-3926361531   2         2         0         6s
nginx-3926361531   2         2         1         18s
nginx-2142116321   1         2         2         2m
nginx-2142116321   1         2         2         2m
nginx-3926361531   3         2         1         18s
nginx-3926361531   3         2         1         18s
nginx-2142116321   1         1         1         2m
nginx-3926361531   3         3         1         18s
nginx-3926361531   3         3         2         19s
nginx-2142116321   0         1         1         2m
nginx-2142116321   0         1         1         2m
nginx-2142116321   0         0         0         2m
nginx-3926361531   3         3         3         20s
^C
$ kubectl get rs
NAME               DESIRED   CURRENT   READY     AGE
nginx-2142116321   0         0         0         2m
nginx-3926361531   3         3         3         28s