阿里巴巴最火的技术：k8s基于Ingress-nginx实现灰度发布
前言
Ingress-Nginx是一个K8S ingress工具，支持配置Ingress Annotations来实现不同场景下的灰度发布和测试。 Nginx Annotations 支持以下几种Canary规则：
Nginx ingress annotations
假设我们现在部署了两个版本的服务，老版本和canary版本
- nginx.ingress.kubernetes.io/canary-by-header：基于Request Header的流量切分，适用于灰度发布以及 A/B 测试。当Request Header 设置为 always时，请求将会被一直发送到 Canary 版本；当 Request Header 设置为 never时，请求不会被发送到 Canary 入口。
- nginx.ingress.kubernetes.io/canary-by-header-value：要匹配的 Request Header 的值，用于通知 Ingress 将请求路由到 Canary Ingress 中指定的服务。当 Request Header 设置为此值时，它将被路由到 Canary 入口。该
- nginx.ingress.kubernetes.io/canary-weight：基于服务权重的流量切分，适用于蓝绿部署，权重范围 0 - 100 按百分比将请求路由到 Canary Ingress 中指定的服务。权重为 0 意味着该金丝雀规则不会向 Canary 入口的服务发送任何请求。权重为60意味着60%流量转到canary。权重为 100 意味着所有请求都将被发送到 Canary 入口。
- nginx.ingress.kubernetes.io/canary-by-cookie：基于 Cookie 的流量切分，适用于灰度发布与 A/B 测试。用于通知 Ingress 将请求路由到 Canary Ingress 中指定的服务的cookie。当 cookie 值设置为 always时，它将被路由到 Canary 入口；当 cookie 值设置为 never时，请求不会被发送到 Canary 入口。
部署服务
这里我们服务的 deployment 就不展示了，service 配置如下
# 测试版本
apiVersion: v1
kind: Service
metadata:
name: hello-service
labels:
app: hello-service
spec:
ports:
- port: 80
protocol: TCP
selector:
app: hello-service
# canary 版本
apiVersion: v1
kind: Service
metadata:
name: canary-hello-service
labels:
app: canary-hello-service
spec:
ports:
- port: 80
protocol: TCP
selector:
app: canary-hello-service根据权重转发
ingress 配置如下：
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: canary
annotations:
kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/canary: "true"
nginx.ingress.kubernetes.io/canary-weight: "30"
spec:
rules:
- host: canary-service.abc.com
http:
paths:
- backend:
serviceName: canary-hello-service
servicePort: 80测试结果如下：
$ for i in $(seq 1 10); do curl http://canary-service.abc.com; echo '\n'; done
hello world-version1
hello world-version1
hello world-version2
hello world-version2
hello world-version1
hello world-version1
hello world-version1
hello world-version1
hello world-version1
hello world-version1根据请求头转发
annotation 配置如下（ingress 其余部分省略）
annotations:
kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/canary: "true"
nginx.ingress.kubernetes.io/canary-by-header: "test"测试结果如下:
$ for i in $(seq 1 5); do curl http://canary-service.abc.com; echo '\n'; done
hello world-version1
hello world-version1
hello world-version1
hello world-version1
hello world-version1
$ for i in $(seq 1 5); do curl -H 'test:always' http://canary-service.abc.com; echo '\n'; done
hello world-version2
hello world-version2
hello world-version2
hello world-version2
hello world-version2根据特定的请求头和值转发
annotation 配置如下:
kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/canary: "true"
nginx.ingress.kubernetes.io/canary-by-header: "test"
nginx.ingress.kubernetes.io/canary-by-header-value: "abc"测试结果如下：
$ for i in $(seq 1 5); do curl -H 'test:always' http://canary-service.abc.com; echo '\n'; done
hello world-version1
hello world-version1
hello world-version1
hello world-version1
hello world-version1
$ for i in $(seq 1 5); do curl -H 'test:abc' http://canary-service.abc.com; echo '\n'; done
hello world-version2
hello world-version2
hello world-version2
hello world-version2
hello world-version2根据 cookie 转发
使用cookie来进行流量管理的场景比较适合用于A/B test，比如用户的请求 cookie 中含有特殊的标签，那么我们可以把这部分用户的请求转发到特定的服务进行处理。annotation 配置如下:
kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/canary: "true"
nginx.ingress.kubernetes.io/canary-by-cookie: "like_music"测试结果如下：
$ for i in $(seq 1 5); do curl -b 'like_music=1' http://canary-service.abc.com; echo '\n'; done
hello world-version1
hello world-version1
hello world-version1
hello world-version1
hello world-version1
$ for i in $(seq 1 5); do curl -b 'like_music=always' http://canary-service.abc.com; echo '\n'; done
hello world-version2
hello world-version2
hello world-version2
hello world-version2
hello world-version2三种annotation按如下顺序匹配
canary-by-header > canary-by-cookie > canary-weight
总结
我们从以上的测试结果来看，可以把流量的转发分成以下两类：
1.根据权重
nginx.ingress.kubernetes.io/canary: "true"
nginx.ingress.kubernetes.io/canary-weight: "30"
70%
|------> 生产版本
users --- 100% ---> Nginx Ingress ----| 30%
|------> canary 版本2.根据用户
nginx.ingress.kubernetes.io/canary: "true"
nginx.ingress.kubernetes.io/canary-by-header: "test"
nginx.ingress.kubernetes.io/canary-by-header-value: "abc"
others
|-----------> 生产版本
users ------> Nginx Ingress ------| "test:abc"
|-----------> canary 版本