k8s基于 ingress-nginx 实现金丝雀发布（灰度发布）

环境

kubernetes v1.15
nginx-ingress-controller v0.32.0

介绍

金丝雀发布：又叫灰度发布，控制产品从A版本平滑的过度到B版本

ingress-nginx：k8s ingress工具，支持金丝雀发布，可以实现基于权重、请求头、请求头的值、cookie转发流量。

rancher：k8s集群管理工具，使用UI简化k8s相关操作

ingress-nginx canary官方说明：//github.com/kubernetes/ingress-nginx/blob/master/docs/user-guide/nginx-configuration/annotations.md#canary

首先创建两个nginx应用

#deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: appv1
  labels:
    app: v1
spec:
  replicas: 1
  selector:
    matchLabels:
      app: v1
  template:
    metadata:
      labels:
        app: v1
    spec:
      containers:
      - name: nginx
        image: zerchin/canary:v1
        ports:
        - containerPort: 80
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: appv2
  labels:
    app: v2
spec:
  replicas: 1
  selector:
    matchLabels:
      app: v2
  template:
    metadata:
      labels:
        app: v2
    spec:
      containers:
      - name: nginx
        image: zerchin/canary:v2
        ports:
        - containerPort: 80

kubectl查看pod

kubectl get pod -o wide |grep app
appv1-69bf545f96-lkqfm            1/1     Running   0          89m    172.17.0.12   minikube              
appv2-787866684-b9hhh             1/1     Running   0          89m    172.17.0.11   minikube              

这两个应用输出以下内容

$ curl 172.17.0.12
v1
$ 
$ curl 172.17.0.11
canary-v2

分别为应用创建对应的service

假设你已经了解了svc 和deployment概念，这步操作你应该了解。

#service.yaml
apiVersion: v1
kind: Service
metadata:
  name: appv1
spec:
  selector:
    app: v1
  ports:
    - protocol: TCP
      port: 80
      targetPort: 80
---
apiVersion: v1
kind: Service
metadata:
  name: appv2
spec:
  selector:
    app: v2
  ports:
    - protocol: TCP
      port: 80
      targetPort: 80

验证svc是否成功运行了

kubectl get svc |grep app
appv1            ClusterIP   10.109.87.198            80/TCP           7m
appv2            ClusterIP   10.106.40.192            80/TCP           7m

部署一个正常的ingress

路由到appv1上

#ingress.yaml
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: app
  namespace: default
spec:
  rules:
  - host: nginx2.192.168.99.101.xip.io
    http:
      paths:
      - backend:
          serviceName: appv1
          servicePort: 80
        path: /

kubectl查看ingress

kubectl get ingress
NAME               HOSTS                          ADDRESS          PORTS   AGE
app                nginx2.192.168.99.101.xip.io   192.168.99.101   80      9m43s

访问nginx2.192.168.99.101.xip.io

curl  nginx2.192.168.99.101.xip.io
###v1

金丝雀发布

有三种方法
金丝雀发布规则优先级：canary-by-header -> canary-by-cookie -> canary-weight
目前，每个ingress规则最多可以应用一个canary ingress
我把yaml文件贴出来，注释掉分别就可以用了。

#canary.yaml
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  annotations:
    kubernetes.io/ingress.class: nginx
    nginx.ingress.kubernetes.io/canary: "true"
# 基于cookie转发流量    nginx.ingress.kubernetes.io/canary-by-cookie: "test"
#  基于请求头转发流量  nginx.ingress.kubernetes.io/canary-by-header: "canary"
#   基于请求头和请求头的值转发流量 nginx.ingress.kubernetes.io/canary-by-header-value: "haha"
  基于权重转发流量  nginx.ingress.kubernetes.io/canary-weight: "30"
  name: app-canary
  namespace: default
spec:
  rules:
  - host: nginx2.192.168.99.101.xip.io
    http:
      paths:
      - backend:
          serviceName: appv2
          servicePort: 80
        path: /

总文件个数.png

1.基于权重转发流量
nginx.ingress.kubernetes.io/canary-by-header：用于通知Ingress将请求路由到Canary Ingress中指定的服务的标头。当请求标头设置always为时，它将被路由到Canary。当标头设置never为时，它将永远不会路由到金丝雀。对于任何其他值，标头将被忽略，并且按优先级将请求与其他金丝雀规则进行比较。

修改app-canary的ingress配置，修改annotation，如下：

  annotations:
    kubernetes.io/ingress.class: nginx
    nginx.ingress.kubernetes.io/canary: "true"
    nginx.ingress.kubernetes.io/canary-by-header: "canary"

结果

for i in `seq 1 10`;do curl http://nginx2.192.168.99.101.xip.io/;done
canary-v2
canary-v2
canary-v2
v1
v1
v1
v1
v1
v1
v1

2.基于请求头转发流量
nginx.ingress.kubernetes.io/canary-by-header：用于通知Ingress将请求路由到Canary Ingress中指定的服务的标头。当请求标头设置always为时，它将被路由到Canary。当标头设置never为时，它将永远不会路由到金丝雀。对于任何其他值，标头将被忽略，并且按优先级将请求与其他金丝雀规则进行比较。

修改app-canary的ingress配置，修改annotation，如下：

  annotations:
    kubernetes.io/ingress.class: nginx
    nginx.ingress.kubernetes.io/canary: "true"
    nginx.ingress.kubernetes.io/canary-by-header: "canary"

测试结果

# curl http://nginx2.192.168.99.101.xip.io/
v1
# curl -H "canary:always" http://nginx2.192.168.99.101.xip.io/
canary-v2

3.基于请求头和请求头的值转发流量
nginx.ingress.kubernetes.io/canary-by-header-value：匹配的报头值，用于通知Ingress将请求路由到Canary Ingress中指定的服务。当请求标头设置为此值时，它将被路由到Canary。对于任何其他标头值，标头将被忽略，并按优先级将请求与其他金丝雀规则进行比较。此注释必须与nginx.ingress.kubernetes.io/canary-by-header一起使用。

修改app-canary的ingress配置，修改annotation，如下：

  annotations:
    kubernetes.io/ingress.class: nginx
    nginx.ingress.kubernetes.io/canary: "true"
    nginx.ingress.kubernetes.io/canary-by-header: "canary"
    nginx.ingress.kubernetes.io/canary-by-header-value: "haha"

测试

# curl http://nginx2.192.168.99.101.xip.io/
v1
# curl -H "canary:haha" http://nginx2.192.168.99.101.xip.io/
canary-v2
# curl -H "canary:always" http://nginx2.192.168.99.101.xip.io/
v1

4.基于cookie转发流量
nginx.ingress.kubernetes.io/canary-by-cookie：用于通知Ingress将请求路由到Canary Ingress中指定的服务的cookie。当cookie值设置always为时，它将被路由到canary。当cookie设置never为时，它将永远不会路由到Canary。对于任何其他值，将忽略cookie，并按优先级将请求与其他canary规则进行比较。

修改app-canary的ingress配置，修改annotation，如下：

  annotations:
    kubernetes.io/ingress.class: nginx
    nginx.ingress.kubernetes.io/canary: "true"
    nginx.ingress.kubernetes.io/canary-by-cookie: "test"

测试结果

# curl  http://nginx2.192.168.99.101.xip.io/
v1
# curl -b "test=never"  http://nginx2.192.168.99.101.xip.io/
v1
# curl -b "test=always"  http://nginx2.192.168.99.101.xip.io/
canary-v2
# curl -b "test=1"  http://nginx2.192.168.99.101.xip.io/
v1

高级的可以参考：
Kubernetes Ingress-nginx使用