rollout

构建 测试 代码扫描 打包 打镜像 推送镜像 部署应用

Rollout 是 Kubernetes 中用于管理部署应用的资源对象。它通过控制多个 ReplicaSets 对象的创建、缩放和更新，以实现无宕机升级和回滚应用程序版本的能力。

一般来说，在 Kubernetes 中，我们部署应用程序时会使用 Deployment 资源对象来创建和管理应用程序副本，Deployment 可以方便地对应用程序进行扩缩容、版本更新和回滚等操作。而 Deployment 实际上是从 ReplicaSet 派生而来的，每个 Deployment 都会创建一个 ReplicaSet。因此，Rollout 除了通过 Deployment 进行版本控制外，还可以直接操作 ReplicaSet。

Rollout 可以通过以下方式进行应用程序版本更新和回滚：

• 通过更新 Deployment 或者直接更新 ReplicaSet 中的 Pod 模板来更新应用程序版本；
• 通过 Deployment 自身的滚动更新机制，逐步替换旧版本 Pod 为新版本 Pod，以达到平稳升级的目的；
• 通过回滚 Deployment 或直接回滚 ReplicaSet，将应用程序回滚到之前的版本。

Rollout 的使用可以避免应用程序在更新过程中出现宕机或数据丢失的问题，帮助用户更好地进行应用程序的管理和维护。

希望能够解答您的问题！如果您还有任何疑问，请随时提问。

Rollout:
Rollout资源
Rollout Controller

https://github.com/argoproj/argo-rollouts
https://argoproj.github.io/argo-rollouts/installation/

部署

第一步：部署Argo Rollouts （Rollouts自身，包括Rollouts控制器和它所提供的几个自定义资源）

kubectl create namespace argo-rollouts
kubectl apply -n argo-rollouts -f https://github.com/argoproj/argo-rollouts/releases/latest/download/install.yaml

查看资源群组

kubectl api-versions

查看某个组内有多少个资源类型
kubectl api-resources --api-group=argoproj.io

第二步：部署Dashboard （图形面板，允许我们用图形化的界面监视它的更新过程，了解更新状态）

kubectl apply -f https://github.com/argoproj/argo-rollouts/releases/download/v1.6.2/dashboard-install.yaml -n argo-rollouts



基于文件定义ingress资源开启Dashboard的外部流量策略（没有认证功能，开放以后都可访问，适合内部使用）
root@k8s-master01:~/learning-jenkins-cicd/09-argocd-and-rollout/01-ingress-argocd-and-rollouts# kubectl apply -f ingress-argo-rollouts.yaml 
ingress.networking.k8s.io/argo-rollouts created


root@k8s-master01:/etc/kubernetes/manifests# kubectl get ingress -n argo-rollouts 
NAME            CLASS   HOSTS                                    ADDRESS   PORTS   AGE
argo-rollouts   nginx   rollouts.magedu.com,rollout.magedu.com             80      36s

windows添加hosts解析
访问

安装
https://github.com/argoproj/argo-rollouts/releases/download/v1.6.2/kubectl-argo-rollouts-linux-amd64

root@k8s-master01:~# install -m 755 kubectl-argo-rollouts-linux-amd64 /usr/local/bin/kubectl-argo-rollouts

建立软连接
root@k8s-master01:~# ln -sv /usr/local/bin/kubectl-argo-rollouts /usr/local/bin/kubectl-rollouts

接下来就可以直接使用了
kubectl rollouts

部署应用
root@k8s-master01:~/learning-jenkins-cicd/09-argocd-and-rollout/rollout-demos# kubectl apply -f 01-basic-rollouts-demo.yaml

cat 01-basic-rollouts-demo.yaml 
# CopyRight: MageEdu 
# 
---
apiVersion: argoproj.io/v1alpha1
kind: Rollout
metadata:
  name: rollouts-spring-boot-helloworld
spec:
  replicas: 10 
  strategy:
    canary:
      steps:
      - setWeight: 10 
      - pause: {}
      - setWeight: 20
      - pause: {duration: 20}
      - setWeight: 30
      - pause: {duration: 20}
      - setWeight: 40
      - pause: {duration: 20}
      - setWeight: 60
      - pause: {duration: 20}
      - setWeight: 80
      - pause: {duration: 20}
  revisionHistoryLimit: 5
  selector:
    matchLabels:
      app: spring-boot-helloworld
  template:
    metadata:
      labels:
        app: spring-boot-helloworld
    spec:
      containers:
      - name: spring-boot-helloworld
        image: ikubernetes/spring-boot-helloworld:v0.9.2
        ports:
        - name: http
          containerPort: 80
          protocol: TCP
        resources:
          requests:
            memory: 32Mi
            cpu: 50m
        livenessProbe:
          httpGet:
            path: '/'
            port: 80
            scheme: HTTP
          initialDelaySeconds: 3
        readinessProbe:
          httpGet:
            path: '/'
            port: 80
            scheme: HTTP
          initialDelaySeconds: 5
---
apiVersion: v1
kind: Service
metadata:
  name: spring-boot-helloworld
spec:
  ports:
  - port: 80
    targetPort: http
    protocol: TCP
    name: http
  selector:
    app: spring-boot-helloworld
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: spring-boot-helloworld
spec:
  ingressClassName: "nginx"
  rules:
    - host: hello.magedu.com
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: spring-boot-helloworld
                port:
                  number: 80
---

查看
http://rollouts.magedu.com/rollouts/default

查看资源
kubectl get rollouts

查看资源具体情况
root@k8s-node01:~# kubectl get rollouts.argoproj.io rollouts-spring-boot-helloworld

-w资源监视

列出rollouts资源
kubectl rollouts list rollouts

对rollouts资源进行更新
kubectl rollouts set image rollouts-spring-boot-helloworld spring-boot-helloworld=harbor.luohw.net/ikubernetes/spring-boot-helloworld
(set image:指定更新哪个资源，并更新资源中的pod容器的image）

可对资源进行持续监视

kubectl rollouts get rollouts rollouts-spring-boot-helloworld -w

请求没有问题，更新版本能够正常提供服务，接下来就可以通过设置图形页面PROMOTE来继续完成后面的更新

查看当前状态发现现在是暂停状态
kubectl rollouts list rollouts
状态为Paused（暂停状态）

使用命令触发后续更新
kubectl rollouts promote rollouts-spring-boot-helloworld

更新完成

argo rollouts结合Ingress Nginx进行Canary流量迁移

Istio环境中支持两种流量分割模式
◼ 更新期间，使用不同的Service分别承载新旧版本的流量
◆Canary和Stable版本分别对应一个独立的Service
⚫ canaryService：待发布的新版本
⚫ stableService：待更新的旧版本
◆分别为Canary和Stable的Pod添加rollouts-pod-template-hash标签，其值为相应的RS模板的hash值
◼ 通过Ingress完成流量分割和迁移
◆更新期间，默认创建的Ingress用于承载旧版本的请求流量
◆另外生成一个专用的Ingress用于承载Canary流量，并将这些流量转发到canaryServer相关的EndPoint之上
◆动态调整canaryIngress和stableIngress的weight进行流量迁移
◼ 更新完后，所有流量回转至stableIngress和stableService，此时相关的后端EndPoint已经切换为发布的新版本

提示：这种Canary期间VS的动态调整可能会导致通过GitOps自动化部署时的问题：权重的瞬时摆动

测试此示例前，先把之前启动的service删除掉
kubectl delete -f 01-basic-rollotes-demo.yaml

查看是否运行rollouts资源
kubectl get rollouts

执行运行命令：
root@k8s-master01:~/learning-jenkins-cicd/09-argocd-and-rollout/rollout-demos# kubectl apply -f 02-rollouts-with-ingress-nginx-traffic-shifting.yaml

查看运行的service，有两个
kubectl get svc
spring-boot-helloworld
spring-boot-helloworld-canary

查看spring-boot-helloworld，会发现自动增添标签选择器的条件,pod的配置改变，其hash值也会改变
kubectl get svc spring-boot-helloworld -o yaml

此时，访问哪一个service都是一样的，一旦启动更新，这两个service就不一样了

查看ingress
kubectl get ingress

更新
root@k8s-master01:~# kubectl rollouts set image rollouts-helloworld-with-traffic-shifting spring-boot-helloworld=ikubernetes/demoapp:v1.1
rollout “rollouts-helloworld-with-traffic-shifting” image updated

Argo Rollouts请求分析和渐进式交付

编辑svc 添加
root@k8s-master01:~/learning-jenkins-cicd/09-argocd-and-rollout/02-metrics-ingress-nginx-controller# kubectl get svc -n ingress-nginx
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
ingress-nginx-controller LoadBalancer 10.96.70.31 192.168.1.201 80:32316/TCP,443:31278/TCP 42d
ingress-nginx-controller-admission ClusterIP 10.102.84.77 443/TCP 42d

root@k8s-master01:~/learning-jenkins-cicd/09-argocd-and-rollout/02-metrics-ingress-nginx-controller# kubectl edit svc ingress-nginx-controller -n ingress-nginx
service/ingress-nginx-controller edited
metadata:
annotations:的下面添加下面二行
prometheus.io/scrape: “true”
prometheus.io/port: “10254”

克隆 部署Prometheus
git clone https://github.com/iKubernetes/k8s-prom.git
root@k8s-master01:~/k8s-prom# kubectl apply -f namespace.yaml
namespace/prom created
root@k8s-master01:~/k8s-prom# kubectl apply -f prometheus/ -n prom

访问： http://prom.magedu.com/

nginx_ingress_controller_requests{service=~"spring-boot-helloworld",namespace="default"}[1m]

查询

访问产生数据

while true ; do curl hello.magedu.com ;sleep 0.8;done

rate(nginx_ingress_controller_requests{service=~"spring-boot-helloworld",namespace="default"}[1m])

求和

sum(rate(nginx_ingress_controller_requests{service=~"spring-boot-helloworld",namespace="default"}[1m]))

分子是正常响应码
分母是全不响应码
产生错误响应码：while true ; do curl  hello.magedu.com/index.hthl  ;sleep 0.8;do
查询
sum(rate(nginx_ingress_controller_requests{service=~"spring-boot-helloworld",namespace="default",status!~"[4-5].*"}[1m]))/sum(rate(nginx_ingress_controller_requests{service=~"spring-boot-helloworld",namespace="default"}[1m]))

删除前面的示例
root@k8s-master01:~/learning-jenkins-cicd/09-argocd-and-rollout/rollout-demos# kubectl delete -f 02-rollouts-with-ingress-nginx-traffic-shifting.yaml

部署03示例
root@k8s-master01:~/learning-jenkins-cicd/09-argocd-and-rollout/rollout-demos# kubectl apply -f 03-rollouts-with-prometheus-analysis.yaml

cat 03-rollouts-with-prometheus-analysis.yaml 
# CopyRight: MageEdu 
#
---
apiVersion: argoproj.io/v1alpha1
kind: AnalysisTemplate
metadata:
  name: success-rate
spec:
  args:
  - name: service-name
  - name: namespace
  metrics:
  - name: success-rate
    # NOTE: prometheus queries return results in the form of a vector.
    # So it is common to access the index 0 of the returned array to obtain the value
    successCondition: result[0] >= 0.95
    interval: 20s 
    count: 5
    failureLimit: 3
    provider:
      prometheus:
        address: http://prometheus.prom.svc.cluster.local:9090
        query: |
          sum(irate(nginx_ingress_controller_requests{service=~"{{args.service-name}}",namespace=~"{{args.namespace}}",status!~"[4-5].*"}[1m])) / 
          sum(irate(nginx_ingress_controller_requests{service=~"{{args.service-name}}",namespace=~"{{args.namespace}}"}[1m]))
---
apiVersion: argoproj.io/v1alpha1
kind: Rollout
metadata:
  name: rollouts-helloworld-with-analysis
spec:
  replicas: 10 
  strategy:
    canary:
      analysis:
        templates:
        - templateName: success-rate
        args:
        - name: service-name
          value: spring-boot-helloworld
        - name: namespace
          value: default
      canaryService: spring-boot-helloworld-canary
      stableService: spring-boot-helloworld
      trafficRouting:
        nginx:
          stableIngress: spring-boot-helloworld
      steps:
      - setCanaryScale:
          matchTrafficWeight: true
      - setWeight: 5
      - pause: {duration: 2m}
      - setWeight: 10
      - pause: {duration: 1m}
      - setWeight: 20
      - pause: {duration: 40}
      - setWeight: 40
      - pause: {duration: 20}
      - setWeight: 60
      - pause: {duration: 20}
      - setWeight: 80
      - pause: {duration: 20}
  revisionHistoryLimit: 5
  selector:
    matchLabels:
      app: spring-boot-helloworld
  template:
    metadata:
      labels:
        app: spring-boot-helloworld
    spec:
      containers:
      - name: spring-boot-helloworld
        image: ikubernetes/spring-boot-helloworld:v0.9.2
        ports:
        - name: http
          containerPort: 80
          protocol: TCP
        resources:
          requests:
            memory: 32Mi
            cpu: 50m
        livenessProbe:
          httpGet:
            path: '/'
            port: 80
            scheme: HTTP
          initialDelaySeconds: 3
        readinessProbe:
          httpGet:
            path: '/'
            port: 80
            scheme: HTTP
          initialDelaySeconds: 5
---
apiVersion: v1
kind: Service
metadata:
  name: spring-boot-helloworld
spec:
  ports:
  - port: 80
    targetPort: http
    protocol: TCP
    name: http
  selector:
    app: spring-boot-helloworld
---
apiVersion: v1
kind: Service
metadata:
  name: spring-boot-helloworld-canary
spec:
  ports:
  - port: 80
    targetPort: http
    protocol: TCP
    name: http
  selector:
    app: spring-boot-helloworld
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: spring-boot-helloworld
spec:
  ingressClassName: "nginx"
  rules:
    - host: hello.magedu.com
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: spring-boot-helloworld
                port:
                  number: 80
---

更新镜像

root@k8s-master01:~/k8s-prom# kubectl get rollouts.argoproj.io 
NAME                                DESIRED   CURRENT   UP-TO-DATE   AVAILABLE   AGE
rollouts-helloworld-with-analysis   10        10        10           7           91s

root@k8s-master01:~/k8s-prom# kubectl rollouts set image rollouts-helloworld-with-analysis spring-boot-helloworld=ikubernetes/demoapp:v1.0
rollout "rollouts-helloworld-with-analysis" image updated

进行分析，查询5次，得平均值，如果满足>0.95得条件，则会自动更新，反之，会回滚

这个错误三次失败

错误过多，进行了回滚