使用k8s-ingress进行金丝雀发布

使用k8s-ingress进行金丝雀发布

灰度发布又叫金丝雀发布，只升级部分服务，即让一部分用户继续用老版本，一部分用户开始用新版本，如果用户对新版本没什么意见，那么逐步扩大范围，把所有用户都迁移到新版本上面来。 (金丝雀发布由来。以前，旷工开矿，在下矿洞前需要检查下方是否有毒气，矿工们先会放一只金丝雀进去探是否有有毒气体，看金丝雀能否活下来。)

环境：

Kubernetes：v1.21.9
Docker：20.10.12

首先创建两个组POD、Service、Ingress

1、旧版本

使用nginx作为容器，使用command命令修改nginx的主页内容。

apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: bill-service-v1
  name: bill-service-v1
  namespace: istio-demo
spec:
  replicas: 1
  selector:
    matchLabels:
      app: bill-service-v1
  template:
    metadata:
      labels:
        app: bill-service-v1
    spec:
      containers:
      - image: nginx:alpine
        name: bill-service-v1
        # 此处修改nginx主页显示为 this is bill-service-v1
        command: ["/bin/sh","-c","echo 'this is bill-service-v1'>/usr/share/nginx/html/index.html;nginx -g 'daemon off;'"]

配置调用v1容器的服务

apiVersion: v1
kind: Service
metadata:
  name: bill-service-svc-v1
  namespace: istio-demo
spec:
  ports:
  - name: http
    port: 80
    protocol: TCP
    targetPort: 80
  selector:
    app: bill-service-v1
  type: ClusterIP

配置Ingress，注意注解 kubernetes.io/ingress.class: "nginx"需要进行填入，表示使用nginx-ingress，官网地址如下： https://kubernetes.github.io/ingress-nginx/

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: gray-release
  namespace: istio-demo
  annotations:
    kubernetes.io/ingress.class: "nginx"
spec:
  rules:
    - host: "bill-service-v1.com"
      http:
        paths:
          - path: "/"
            pathType: Prefix
            backend:
              service:
                # 指定服务的名称和端口
                name: bill-service-svc-v1
                port:
                  number: 80

把以上配置使用 kubectl apply -f，进行应用

查看ingress-controller安装的地址，我的机器上是 192.168.80.69

[root@k8smaster canaryBird]# kubectl get pods -n ingress-nginx -o wide
NAME                                        READY   STATUS      RESTARTS   AGE   IP              NODE       NOMINATED NODE   READINESS GATES
ingress-nginx-controller-6db5d74b49-8mdnw   0/1     NodePorts   0          24d             k8snode1              
ingress-nginx-controller-6db5d74b49-bn6gt   1/1     Running     3          24d   192.168.80.69   k8snode1              
ingress-nginx-controller-6db5d74b49-bqd8v   0/1     NodePorts   0          24d             k8snode1              
ingress-nginx-controller-6db5d74b49-z58gw   0/1     NodePorts   0          24d             k8snode1              

配置host(我是用的是widows)，可以使用软件SwitchHosts进行调整

192.168.80.69 bill-service-v1.com

在配置host的机器上访问地址 bill-service-v1.com，显示

this is bill-service-v1

2、新版本

配置v2版本的负载，表示新发布的项目版本

apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: bill-service-v2
  name: bill-service-v2
  namespace: istio-demo
spec:
  replicas: 1
  selector:
    matchLabels:
      app: bill-service-v2
  template:
    metadata:
      labels:
        app: bill-service-v2
    spec:
      containers:
      - image: nginx:alpine
        name: bill-service-v2
        command: ["/bin/sh","-c","echo 'this is bill-service-v2'>/usr/share/nginx/html/index.html;nginx -g 'daemon off;'"]

创建访问v2版本的service

apiVersion: v1
kind: Service
metadata:
  name: bill-service-svc-v2
  namespace: istio-demo
spec:
  ports:
  - name: http
    port: 80
    protocol: TCP
    targetPort: 80
  selector:
    app: bill-service-v2
  type: ClusterIP

配置v2版本的ingress，注意hos要保持一直，因为我们统一的网址进行访问

v2版本需要添加两个注解：

nginx.ingress.kubernetes.io/canary：是否启用金丝雀功能

nginx.ingress.kubernetes.io/canary-weight：分配流量百分比

具体可以参考地址：https://kubernetes.github.io/ingress-nginx/user-guide/nginx-configuration/annotations/#canary

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: gray-release-v2
  namespace: istio-demo
  annotations:
    kubernetes.io/ingress.class: "nginx"
    nginx.ingress.kubernetes.io/canary: "true"
    nginx.ingress.kubernetes.io/canary-weight: "10"
spec:
  rules:
    - host: "bill-service-v1.com"
      http:
        paths:
          - path: "/"
            pathType: Prefix
            backend:
              service:
                name: bill-service-svc-v2
                port:
                  number: 80

3、测试

这里使用linux进行测试访问情况，可以不需要配置host，直接在命令行中带入，具体命令和输出结果如下

[root@k8smaster canaryBird]# for i in $(seq 1 10);do curl -H "Host: bill-service-v1.com" http://192.168.80.69;echo "";done
this is bill-service-v1

this is bill-service-v1

this is bill-service-v1

this is bill-service-v1

this is bill-service-v1

this is bill-service-v1

this is bill-service-v2

this is bill-service-v1

this is bill-service-v1

this is bill-service-v1

可以看到流量是 10/90来进行分配的，不过这个不是绝对的，大致是这个比例。

4、情况1

如果金丝雀有问题，可以直接使用命令删除金丝雀版本

kubectl delete -f v2.yaml

使用命令进行测试，发现全部会在v1版本上

[root@k8smaster canaryBird]# for i in $(seq 1 10);do curl -H "Host: bill-service-v1.com" http://192.168.80.69;echo "";done
this is bill-service-v1
this is bill-service-v1
this is bill-service-v1
this is bill-service-v1
this is bill-service-v1
this is bill-service-v1
this is bill-service-v1
this is bill-service-v1
this is bill-service-v1
this is bill-service-v1

5、情况2

如果项目运转良好，可以加大流量进行测试，可以直接修改v2-ingress流量为90%

nginx.ingress.kubernetes.io/canary-weight: "90"

应用这些配置

kubectl apply -f v2.yaml

进行测试

[root@k8smaster canaryBird]# for i in $(seq 1 10);do curl -H "Host: bill-service-v1.com" http://192.168.80.69;echo "";done
this is bill-service-v2
this is bill-service-v2
this is bill-service-v2
this is bill-service-v2
this is bill-service-v2
this is bill-service-v2
this is bill-service-v2
this is bill-service-v2
this is bill-service-v2
this is bill-service-v2

可以删除新版本的服务，旧版服务也还是必须存在，不然会报错。