Kubernetes集群 服务暴露 Traefik
Kubernetes集群 服务暴露 Traefik
- 一、认识traefik
-
- 1.1 traefik简介
- 1.2 traefix 特性
- 1.3 traefik与nginx ingress对比
- 1.4 traefik核心概念及能力
- 二、traefik部署
-
- 2.1 获取traefik部署前置资源清单文件
-
- 2.1.1 创建CRD资源
- vim traefik-crd.yaml
-
-
- 2.1.2 创建RBAC权限
- 2.1.3 创建traefik配置文件
- 2.1.4 应用上述资源清单文件
- 2.1.5 设置节点Label
- 2.2 部署Traefik
-
- 2.2.1 deploy资源清单文件准备
- 2.2.2 应用deploy资源清单文件
- 2.2.3 service资源清单文件准备
- 2.2.4 应用service资源清单文件
- 2.3 配置访问traefik dashboard路由规则
-
- 2.3.1 Traefik创建路由规则方法
- 2.3.2 通过原生ingress方式暴露traefik dashboard
- 2.3.3 创建dashboard ingress route资源清单文件
- 2.3.4 应用资源清单文件
- 2.3.5 在集群内或集群外主机配置域名解析
- 2.3.6 通过域名访问traefik dashboard
-
- 三、traefik基础应用
-
- 3.1 配置kubernetes dashboard路由规则
-
- 3.1.1 查看kubernetes dashboard service状态
- 3.1.2 编写访问kubernetes dashboard 路由规则
- 3.1.3 应用上述路由规则文件
- 3.1.4 配置域名解析及访问
- 3.2 配置HTTP路由规则
-
- 3.2.1 创建应用及服务资源清单文件并应用
- 3.2.2 创建whoami应用ingress route资源清单文件并应用
- 3.3 配置HTTPS路由规则
-
- 3.3.1 自签名证书
- 3.3.2 创建secret
- 3.3.3 创建https应用路由规则
- 3.4 配置TCP路由规则
-
- 3.4.1 实验案例配置
- 3.4.2 生产案例配置 MySQL部署及traefik代理
-
- 3.4.2.1 修改traefik-configmap.yaml
- 3.4.2.2 修改traefik-deploy.yaml
- 3.4.2.3 部署mysql应用
- 3.4.2.4 为mysql应用创建ingressroute
- 3.4.2.5 验证
- 3.4.3 生产案例 Redis部署及traefik代理
- 3.5 配置UDP路由规则
- 四、traefik中间件 MiddleWare
-
- 4.1 traefik中间件介绍 MiddleWare
- 4.2 traefik 中间件应用案例 ipWhiteList
- 五、traefik高级应用
-
- 5.1 负载均衡
-
- 5.1.1 创建deployment控制器类型应用
- 5.1.2 创建service
- 5.1.3 创建ingressroute
- 5.1.4 访问验证
- 5.2 灰度发布
-
- 5.2.1 创建TraefikService
- 5.2.2 创建ingressroute
- 5.3 流量复制
-
- 5.3.1 指定流量来自己于kubernetes service对象
- 5.3.2 通过traefikservice导入流量
- 5.3.3 流量复制小结
- 六、Kubernetes Gateway API
-
- 6.1 Gateway API介绍
-
- 6.1.1 Gateway API架构
- 6.1.2 面向角色设计
- 6.1.3 Gateway API概念
-
- 6.1.3.1 GatewayClass
- 6.1.3.2 Gateway
- 6.1.3.3 Route 资源
-
- 6.1.3.3.1 HTTPRoute
- 6.1.3.3.2 TLSRoute
- 6.1.3.3.3 TCPRoute 和 UDPRoute
- 6.1.4 Gateway API概念之间关系
- 6.2 kubernetes gateway CRD安装
- 6.3 为traefik授权（RBAC）
- 6.4 Traefik开启Gateway api支持
- 6.5 创建Gateway API的GatewayClass
- 6.6 Gateway API应用案例
-
- 6.6.1 通过Gateway API方式暴露traefik dashboard
-
- 6.6.1.1 创建gateway
- 6.5.1.2 创建HTTPRoute
- 6.5.1.3 在集群之外主机访问
- 6.6.2 通过Gateway API方式暴露WEB应用
- 6.6.3 金丝雀发布
一、认识traefik
1.1 traefik简介
- 参考链接: https://traefik.cn/
- 是一个为了让部署微服务更加便捷而诞生的现代HTTP反向代理、负载均衡工具。
- 它支持多种后台 (Docker, Swarm, Kubernetes, Marathon, Mesos, Consul, Etcd, Zookeeper, BoltDB, Rest API, file…) 来自动化、动态的应用它的配置文件设置。
1.2 traefix 特性
- 非常快
- 无需安装其他依赖，通过Go语言编写的单一可执行文件
- 支持 Rest API
- 多种后台支持：Docker, Swarm, Kubernetes, Marathon, Mesos, Consul, Etcd, 并且还会更多
- 后台监控, 可以监听后台变化进而自动化应用新的配置文件设置
- 配置文件热更新。无需重启进程
- 正常结束http连接
- 后端断路器
- 轮询，rebalancer 负载均衡
- Rest Metrics
- 支持最小化官方docker 镜像
- 前、后台支持SSL
- 清爽的AngularJS前端页面
- 支持Websocket
- 支持HTTP/2
- 网络错误重试
- 支持Let’s Encrypt (自动更新HTTPS证书)
- 高可用集群模式
1.3 traefik与nginx ingress对比
1.4 traefik核心概念及能力
Traefik是一个边缘路由器，它会拦截外部的请求并根据逻辑规则选择不同的操作方式，这些规则决定着这些请求到底该如何处理。Traefik提供自动发现能力，会实时检测服务，并自动更新路由规则。
从上图可知，请求首先会连接到entrypoints，然后分析这些请求是否与定义的rules匹配，如果匹配，则会通过一系列middlewares，再到对应的services上。
这就涉及到以下几个重要的核心组件。
- Providers
- Entrypoints
- Routers
- Services
- Middlewares
下面分别介绍一下：
- Providers
Providers是基础组件，Traefik的配置发现是通过它来实现的，它可以是协调器，容器引擎，云提供商或者键值存储。
Traefik通过查询Providers的API来查询路由的相关信息，一旦检测到变化，就会动态的更新路由。
-
Entrypoints
Entrypoints是Traefik的网络入口，它定义接收请求的接口，以及是否监听TCP或者UDP。 -
Routers
Routers主要用于分析请求，并负责将这些请求连接到对应的服务上去，在这个过程中，Routers还可以使用Middlewares来更新请求，比如在把请求发到服务之前添加一些Headers。 -
Services
Services负责配置如何到达最终将处理传入请求的实际服务。 -
Middlewares
Middlewares用来修改请求或者根据请求来做出一些判断（authentication, rate limiting, headers, …），中间件被附件到路由上，是一种在请求发送到你的服务之前（或者在服务的响应发送到客户端之前）调整请求的一种方法。
二、traefik部署
2.1 获取traefik部署前置资源清单文件
2.1.1 创建CRD资源
此yaml资源清单文件可在traefik.io网站直接复制使用：https://doc.traefik.io/traefik/v2.5/reference/dynamic-configuration/kubernetes-crd/#definitions
vim traefik-crd.yaml
2.1.2 创建RBAC权限
基于角色的访问控制（RBAC）策略，方便对Kubernetes资源和API进行细粒度控制
traefik需要一定的权限，需要提前创建ServiceAccount并分配一定的权限。
# vim traefik-rbac.yaml
apiVersion: v1
kind: ServiceAccount
metadata:
namespace: kube-system
name: traefik-ingress-controller
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: traefik-ingress-controller
rules:
- apiGroups:
- ""
resources:
- services
- endpoints
- secrets
verbs:
- get
- list
- watch
- apiGroups:
- extensions
- networking.k8s.io
resources:
- ingresses
- ingressclasses
verbs:
- get
- list
- watch
- apiGroups:
- extensions
resources:
- ingresses/status
verbs:
- update
- apiGroups:
- traefik.containo.us
resources:
- middlewares
- middlewaretcps
- ingressroutes
- traefikservices
- ingressroutetcps
- ingressrouteudps
- tlsoptions
- tlsstores
- serverstransports
verbs:
- get
- list
- watch
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: traefik-ingress-controller
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: traefik-ingress-controller
subjects:
- kind: ServiceAccount
name: traefik-ingress-controller
namespace: kube-system
2.1.3 创建traefik配置文件
由traefik配置很多，通过CLI定义不方便，一般都通过配置文件对traefik进行参数配置，例如使用ConfigMap将配置挂载至traefik中
# vim traefik-configmap.yaml
apiVersion: v1
kind: ConfigMap
metadata:
name: traefik
namespace: kube-system
data:
traefik.yaml: |-
serversTransport:
insecureSkipVerify: true ## 略验证代理服务的 TLS 证书
api:
insecure: true ## 允许 HTTP 方式访问 API
dashboard: true ## 启用 Dashboard
debug: true ## 启用 Debug 调试模式
metrics:
prometheus: "" ## 配置 Prometheus 监控指标数据，并使用默认配置
entryPoints:
web:
address: ":80" ## 配置 80 端口，并设置入口名称为 web
websecure:
address: ":443" ## 配置 443 端口，并设置入口名称为 websecure
udpep:
address: ":8084/udp" ## 配置 8084端口，并设置入口名称为 udpep，做为udp入口
providers:
kubernetesCRD: "" ## 启用 Kubernetes CRD 方式来配置路由规则
kubernetesingress: "" ## 启用 Kubernetes Ingress 方式来配置路由规则
kubernetesGateway: "" ## 启用 Kubernetes Gateway API
experimental:
kubernetesGateway: true ## 允许使用 Kubernetes Gateway API
log:
filePath: "" ## 设置调试日志文件存储路径，如果为空则输出到控制台
level: error ## 设置调试日志级别
format: json ## 设置调试日志格式
accessLog:
filePath: "" ## 设置访问日志文件存储路径，如果为空则输出到控制台
format: json ## 设置访问调试日志格式
bufferingSize: 0 ## 设置访问日志缓存行数
filters:
retryAttempts: true ## 设置代理访问重试失败时，保留访问日志
minDuration: 20 ## 设置保留请求时间超过指定持续时间的访问日志
fields: ## 设置访问日志中的字段是否保留（keep 保留、drop 不保留）
defaultMode: keep ## 设置默认保留访问日志字段
names:
ClientUsername: drop
headers:
defaultMode: keep ## 设置 Header 中字段是否保留,设置默认保留 Header 中字段
names: ## 针对 Header 中特别字段特别配置保留模式
User-Agent: redact
Authorization: drop
Content-Type: keep
2.1.4 应用上述资源清单文件
# kubectl apply -f traefik-crd.yaml
# kubectl apply -f traefik-rbac.yaml
# kubectl apply -f traefik-configmap.yaml
2.1.5 设置节点Label
由于使用DaemonSet方式部署Traefik，所以需要为节点设置label，当应用部署时会根据节点Label进行选择。
设置节点标签
# kubectl label nodes --all IngressProxy=true
查看节点标签
# kubectl get nodes --show-labels
如需要取消时，可执行下述命令
# kubectl label nodes --all IngressProxy-
2.2 部署Traefik
2.2.1 deploy资源清单文件准备
本次将用Daemonset方式部署traefik,便于后期扩展
本次部署通过hostport方式把Pod中容器内的80、443映射到物理机，方便集群外访问。
# vim traefik-deploy.yaml
apiVersion: apps/v1
kind: DaemonSet
metadata:
namespace: kube-system
name: traefik
labels:
app: traefik
spec:
selector:
matchLabels:
app: traefik
template:
metadata:
labels:
app: traefik
spec:
serviceAccountName: traefik-ingress-controller
containers:
- name: traefik
image: traefik:v2.5.7
args:
- --configfile=/config/traefik.yaml
volumeMounts:
- mountPath: /config
name: config
ports:
- name: web
containerPort: 80
hostPort: 80 ## 将容器端口绑定所在服务器的 80 端口
- name: websecure
containerPort: 443
hostPort: 443 ## 将容器端口绑定所在服务器的 443 端口
- name: admin
containerPort: 8080 ## Traefik Dashboard 端口
volumes:
- name: config
configMap:
name: traefik
tolerations: ## 设置容忍所有污点，防止节点被设置污点
- operator: "Exists"
nodeSelector: ## 设置node筛选器，在特定label的节点上启动
IngressProxy: "true"
验证端口配置是否正确
# kubectl get daemonset traefik -n kube-system -o yaml
2.2.2 应用deploy资源清单文件
# kubectl apply -f traefix-deploy.yaml
2.2.3 service资源清单文件准备
# vim traefix-service.yaml
apiVersion: v1
kind: Service
metadata:
name: traefik
namespace: kube-system
spec:
ports:
- protocol: TCP
name: web
port: 80
- protocol: TCP
name: admin
port: 8080
- protocol: TCP
name: websecure
port: 443
selector:
app: traefik
2.2.4 应用service资源清单文件
# kubectl apply -f traefik-service.yaml
2.3 配置访问traefik dashboard路由规则
Traefik 应用已经部署完成，但是想让外部访问 Kubernetes 内部服务，还需要配置路由规则，上面部署 Traefik 时开启了 Traefik Dashboard，这是 Traefik 提供的视图看板，所以，首先配置基于 HTTP 的 Traefik Dashboard 路由规则，使外部能够访问 Traefik Dashboard。这里使用 IngressRoute方式进行演示。
2.3.1 Traefik创建路由规则方法
- 原生ingress
- CRD IngressRoute
- Gateway API
2.3.2 通过原生ingress方式暴露traefik dashboard
# kubectl get svc -n kube-system
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
traefik ClusterIP 10.96.174.88 80/TCP,8080/TCP,443/TCP 44m
# kubectl get endpoints -n kube-system
NAME ENDPOINTS AGE
traefik 10.244.135.204:80,10.244.159.141:80,10.244.194.92:80 + 17 more... 44m
# cat traefik-dashboard-native-ingress.yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: traefik-dashboard-ingress
namespace: kube-system
annotations:
kubernetes.io/ingress.class: traefik
traefik.ingress.kubernetes.io/router.entrypoints: web
spec:
rules:
- host: tfni.test.com
http:
paths:
- pathType: Prefix
path: /
backend:
service:
name: traefik
port:
number: 8080
# kubectl apply -f traefik-dashboard-native-ingress.yaml
# kubectl get ingress -n kube-system
NAME CLASS HOSTS ADDRESS PORTS AGE
traefik-dashboard-ingress tfni.test.com 80 56m
# kubectl describe ingress traefik-dashboard-ingress -n kube-system
Name: traefik-dashboard-ingress
Namespace: kube-system
Address:
Default backend: default-http-backend:80 ("default-http-backend" not found>)
Rules:
Host Path Backends
---- ---- --------
tfni.test.com
/ traefik:8080 (10.244.135.204:8080,10.244.159.141:8080,10.244.194.92:8080 + 1 more...)
Annotations: kubernetes.io/ingress.class: traefik
traefik.ingress.kubernetes.io/router.entrypoints: web
Events:
在集群之外主机访问
# vim /etc/hosts
......
192.168.10.12 tfni.test.com
2.3.3 创建dashboard ingress route资源清单文件
# vim traefik-dashboard-ingress-route.yaml
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
name: traefik
namespace: kube-system
spec:
entryPoints:
- web
routes:
- match: Host(`traefik.test.com`) && PathPrefix(`/`)
kind: Rule
services:
- name: traefik
port: 8080
2.3.4 应用资源清单文件
# kubectl apply -f traefik-dashboard-ingress-route.yaml
2.3.5 在集群内或集群外主机配置域名解析
把域名解析为kubernetes集群任意节点IP既可。
# vim /etc/hosts
192.168.10.12 traefik.test.com
2.3.6 通过域名访问traefik dashboard
# firefox http://traefik.test.com &
三、traefik基础应用
3.1 配置kubernetes dashboard路由规则
必须使用SSL证书创建secret密钥
3.1.1 查看kubernetes dashboard service状态
# kubectl get svc -n kubernetes-dashboard
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
dashboard-metrics-scraper ClusterIP 10.96.100.4 8000/TCP 21d
kubernetes-dashboard NodePort 10.96.19.1 443:30000/TCP 21d
3.1.2 编写访问kubernetes dashboard 路由规则
# kubectl create secret tls kubemsb-tls --cert=kubemsb.pem --key=kubemsb.key
# kubectl get secret
NAME TYPE DATA AGE
default-token-x4qbc kubernetes.io/service-account-token 3 24d
kubemsb-tls kubernetes.io/tls 2 11s
# vim kubernetes-dashboard-ir.yaml
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
name: kubernetes-dashboard
namespace: kubernetes-dashboard
spec:
entryPoints:
- websecure
routes:
- match: Host(`dashboard.test.com`)
kind: Rule
services:
- name: kubernetes-dashboard
port: 443
tls:
secretName: kubemsb-tls
3.1.3 应用上述路由规则文件
# kubectl apply -f kubernetes-dashboard-ir.yaml
3.1.4 配置域名解析及访问
# vim /etc/hosts
192.168.10.12 dashboard.test.com
# kubectl get secret -n kubernetes-dashboard
NAME TYPE DATA AGE
......
kubernetes-dashboard-token-8tm5n kubernetes.io/service-account-token 3 21d
# kubectl describe secret kubernetes-dashboard-token-8tm5n -n kubernetes-dashboard
Name: kubernetes-dashboard-token-8tm5n
Namespace: kubernetes-dashboard
Labels:
Annotations: kubernetes.io/service-account.name: kubernetes-dashboard
kubernetes.io/service-account.uid: 47d19e14-e3ed-4733-9799-02396dfb436a
Type: kubernetes.io/service-account-token
Data
====
ca.crt: 1359 bytes
namespace: 20 bytes
token: jZWFjY291bnQvc2VjcmV0Lm5hbWUiOiJrdWJlcm5ldGVzLWRhc2hib2FyZC10b2tlbi04dG01
3.2 配置HTTP路由规则
3.2.1 创建应用及服务资源清单文件并应用
# vim whoami.yaml
kind: Deployment
apiVersion: apps/v1
metadata:
name: whoami
namespace: default
labels:
app: traefiklabs
name: whoami
spec:
replicas: 2
selector:
matchLabels:
app: traefiklabs
task: whoami
template:
metadata:
labels:
app: traefiklabs
task: whoami
spec:
containers:
- name: whoami
image: traefik/whoami
ports:
- containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
name: whoami
namespace: default
spec:
ports:
- name: http
port: 80
selector:
app: traefiklabs
task: whoami
---
kind: Deployment
apiVersion: apps/v1
metadata:
name: whoamitcp
namespace: default
labels:
app: traefiklabs
name: whoamitcp
spec:
replicas: 2
selector:
matchLabels:
app: traefiklabs
task: whoamitcp
template:
metadata:
labels:
app: traefiklabs
task: whoamitcp
spec:
containers:
- name: whoamitcp
image: traefik/whoamitcp
ports:
- containerPort: 8080
---
apiVersion: v1
kind: Service
metadata:
name: whoamitcp
namespace: default
spec:
ports:
- protocol: TCP
port: 8080
selector:
app: traefiklabs
task: whoamitcp
---
kind: Deployment
apiVersion: apps/v1
metadata:
name: whoamiudp
namespace: default
labels:
app: traefiklabs
name: whoamiudp
spec:
replicas: 2
selector:
matchLabels:
app: traefiklabs
task: whoamiudp
template:
metadata:
labels:
app: traefiklabs
task: whoamiudp
spec:
containers:
- name: whoamiudp
image: traefik/whoamiudp:latest
ports:
- containerPort: 8080
---
apiVersion: v1
kind: Service
metadata:
name: whoamiudp
namespace: default
spec:
ports:
- port: 8080
protocol: UDP
selector:
app: traefiklabs
task: whoamiudp
# kubectl apply -f whoami.yaml
# kubectl get all
NAME READY STATUS RESTARTS AGE
pod/whoami-7d666f84d8-ffbdv 1/1 Running 0 35m
pod/whoami-7d666f84d8-n75wb 1/1 Running 0 35m
pod/whoamitcp-744cc4b47-g98fv 1/1 Running 0 35m
pod/whoamitcp-744cc4b47-s2m7h 1/1 Running 0 35m
pod/whoamiudp-58f6cf7b8-54552 1/1 Running 0 35m
pod/whoamiudp-58f6cf7b8-dzxpg 1/1 Running 0 35m
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/kubernetes ClusterIP 10.96.0.1 443/TCP 25d
service/nginx-metallb LoadBalancer 10.96.109.6 192.168.10.90 8090:30259/TCP 14d
service/whoami ClusterIP 10.96.251.213 80/TCP 35m
service/whoamitcp ClusterIP 10.96.20.1 8080/TCP 35m
service/whoamiudp ClusterIP 10.96.85.175 8080/UDP 35m
NAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/whoami 2/2 2 2 35m
deployment.apps/whoamitcp 2/2 2 2 35m
deployment.apps/whoamiudp 2/2 2 2 35m
NAME DESIRED CURRENT READY AGE
replicaset.apps/whoami-7d666f84d8 2 2 2 35m
replicaset.apps/whoamitcp-744cc4b47 2 2 2 35m
replicaset.apps/whoamiudp-58f6cf7b8 2 2 2 35m
3.2.2 创建whoami应用ingress route资源清单文件并应用
# vim whoami-ingressroute.yaml
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
name: myingressroute
namespace: default
spec:
entryPoints:
- web
routes:
- match: Host(`whoami.test.com`) && PathPrefix(`/`)
kind: Rule
services:
- name: whoami
port: 80
# kubectl apply -f whoami-ingressroute.yaml
# kubectl get ingressroute
NAME AGE
myingressroute 29m
3.3 配置HTTPS路由规则
如果我们需要用 HTTPS 来访问我们这个应用的话，就需要监听 websecure 这个入口点，也就是通过 443 端口来访问，同样用 HTTPS 访问应用必然就需要证书
3.3.1 自签名证书
# openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout tls.key -out tls.crt -subj "/CN=whoamissl.kubemsb.com"
3.3.2 创建secret
# kubectl create secret tls who-tls --cert=tls.crt --key=tls.key
3.3.3 创建https应用路由规则
# vim whoamissl-ingressroute.yaml
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
name: ingressroutetls
spec:
entryPoints:
- websecure
routes:
- match: Host(`whoamissl.test.com`)
kind: Rule
services:
- name: whoami
port: 80
tls:
secretName: who-tls
# kubectl apply -f whoamissl-ingressroute.yaml
# kubectl get ingressroute
NAME AGE
ingressroutetls 17s
# cat /etc/hosts
192.168.10.12 whoamissl.test.com
3.4 配置TCP路由规则
SNI为服务名称标识，是 TLS 协议的扩展。因此，只有 TLS 路由才能使用该规则指定域名。但是，非 TLS 路由必须使用带有
*的规则（每个域）来声明每个非 TLS 请求都将由路由进行处理。
3.4.1 实验案例配置
# vim whoami-ingressroutetcp.yaml
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRouteTCP
metadata:
name: ingressroutetcpwho
spec:
entryPoints:
- tcpep
routes:
- match: HostSNI(`*`)
services:
- name: whoamitcp
port: 8080
# kubectl get ingressroutetcp
NAME AGE
ingressroutetcpwho 17s
3.4.2 生产案例配置 MySQL部署及traefik代理
3.4.2.1 修改traefik-configmap.yaml
# cat traefik-configmap.yaml
apiVersion: v1
kind: ConfigMap
metadata:
name: traefik
namespace: kube-system
data:
traefik.yaml: |-
serversTransport:
insecureSkipVerify: true
api:
insecure: true
dashboard: true
debug: true
metrics:
prometheus: ""
entryPoints:
web:
address: ":80"
websecure:
address: ":443"
metrics:
address: ":8082"
mysql:
address: ":3312"
providers:
kubernetesCRD: ""
kubernetesingress: ""
log:
filePath: ""
level: error
format: json
accessLog:
filePath: ""
format: json
bufferingSize: 0
filters:
retryAttempts: true
minDuration: 20
fields:
defaultMode: keep
names:
ClientUsername: drop
headers:
defaultMode: keep
names:
User-Agent: redact
Authorization: drop
Content-Type: keep
3.4.2.2 修改traefik-deploy.yaml
# vim traefik-deploy.yaml
apiVersion: apps/v1
kind: DaemonSet
metadata:
namespace: kube-system
name: traefik
labels:
app: traefik
spec:
selector:
matchLabels:
app: traefik
template:
metadata:
labels:
app: traefik
spec:
serviceAccountName: traefik-ingress-controller
containers:
- name: traefik
image: traefik:v2.5.7
args:
- --configfile=/config/traefik.yaml
volumeMounts:
- mountPath: /config
name: config
ports:
- name: web
containerPort: 80
hostPort: 80
- name: websecure
containerPort: 443
hostPort: 443
- name: admin
containerPort: 8080
- name: mysql
containerPort: 3312
hostPort: 3306
volumes:
- name: config
configMap:
name: traefik
# vim traefik-service.yaml
apiVersion: v1
kind: Service
metadata:
name: traefik
namespace: kube-system
spec:
ports:
- protocol: TCP
name: web
port: 80
- protocol: TCP
name: admin
port: 8080
- protocol: TCP
name: websecure
port: 443
- protocol: TCP
name: mysql
port: 3312
selector:
app: traefik
删除以前的配置及应用
# kubectl delete -f traefix-configmap.yaml
# kubectl delete -f traefix-deploy.yaml
# kubectl delete -f traefix-service.yaml
重新部署
# kubectl apply -f traefix-configmap.yaml
# kubectl apply -f traefix-deploy.yaml
# kubectl apply -f traefix-service.yaml
3.4.2.3 部署mysql应用
关于端口的说明：
traefik Pod：3312:3306（traefik Pod:k8s Node），3312是traefik配置的mysql入口点的端口，3306是k8s Node的端口，traefik请求入口
# vim mysql-configmap.yaml
apiVersion: v1
kind: ConfigMap
metadata:
name: mysql
labels:
app: mysql
namespace: default
data:
my.cnf: |
[mysqld]
character-set-server = utf8mb4
collation-server = utf8mb4_unicode_ci
skip-character-set-client-handshake = 1
default-storage-engine = INNODB
max_allowed_packet = 500M
explicit_defaults_for_timestamp = 1
long_query_time = 10
# vim mysql-deploy.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
app: mysql
name: mysql
namespace: default
spec:
selector:
matchLabels:
app: mysql
template:
metadata:
labels:
app: mysql
spec:
containers:
- name: mysql
image: mysql:5.7
imagePullPolicy: IfNotPresent
env:
- name: MYSQL_ROOT_PASSWORD
value: abc123
ports:
- containerPort: 3306
volumeMounts:
- mountPath: /var/lib/mysql
name: pv
- mountPath: /etc/mysql/conf.d/my.cnf
subPath: my.cnf
name: cm
volumes:
- name: pv
hostPath:
path: /opt/mysqldata
- name: cm
configMap:
name: mysql
# vim mysql-service.yaml
apiVersion: v1
kind: Service
metadata:
name: mysql
namespace: default
spec:
ports:
- port: 3306
protocol: TCP
targetPort: 3306
selector:
app: mysql
# kubectl apply -f mysql-configmap.yaml
# kubectl apply -f mysql-deploy.yaml
# kubectl apply -f mysql-service.yaml
3.4.2.4 为mysql应用创建ingressroute
# vim mysql-ingressroutetcp.yaml
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRouteTCP
metadata:
name: mysql
namespace: default
spec:
entryPoints:
- mysql
routes:
- match: HostSNI(`*`)
services:
- name: mysql
port: 3306
# kubectl apply -f mysql-ingressroutetcp.yaml
# kubectl get ingressroutetcp
NAME AGE
mysql 8s
3.4.2.5 验证
在集群之外主机上执行如下操作
# cat /etc/hosts
......
192.168.10.12 mysql.test.com
# mysql -h mysql.test.com -uroot -pabc123 -P3306
mysql: [Warning] Using a password on the command line interface can be insecure.
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 3
Server version: 5.7.36 MySQL Community Server (GPL)
Copyright (c) 2000, 2022, Oracle and/or its affiliates.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
mysql>
3.4.3 生产案例 Redis部署及traefik代理
# vim redis.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: redis
namespace: default
spec:
selector:
matchLabels:
app: redis
template:
metadata:
labels:
app: redis
spec:
containers:
- name: redis
image: redis:6.2.6
ports:
- containerPort: 6379
protocol: TCP
---
apiVersion: v1
kind: Service
metadata:
name: redis
namespace: default
spec:
ports:
- port: 6379
targetPort: 6379
selector:
app: redis
# kubectl apply -f redis.yaml
deployment.apps/redis created
service/redis created
# vim traefik-configmap.yaml
apiVersion: v1
kind: ConfigMap
metadata:
name: traefik
namespace: kube-system
data:
traefik.yaml: |-
serversTransport:
insecureSkipVerify: true
api:
insecure: true
dashboard: true
debug: true
metrics:
prometheus: ""
entryPoints:
web:
address: ":80"
websecure:
address: ":443"
metrics:
address: ":8082"
mysql:
address: ":3312"
redis:
address: ":6379"
providers:
kubernetesCRD: ""
kubernetesingress: ""
log:
filePath: ""
level: error
format: json
accessLog:
filePath: ""
format: json
bufferingSize: 0
filters:
retryAttempts: true
minDuration: 20
fields:
defaultMode: keep
names:
ClientUsername: drop
headers:
defaultMode: keep
names:
User-Agent: redact
Authorization: drop
Content-Type: keep
删除原configmap,再重新创建
# kubectl delete -f traefik-configmap.yaml
# kubectl apply -f traefik-configmap.yaml
# vim traefix-deploy.yaml
apiVersion: apps/v1
kind: DaemonSet
metadata:
namespace: kube-system
name: traefik
labels:
app: traefik
spec:
selector:
matchLabels:
app: traefik
template:
metadata:
labels:
app: traefik
spec:
serviceAccountName: traefik-ingress-controller
containers:
- name: traefik
image: traefik:v2.5.7
args:
- --configfile=/config/traefik.yaml
volumeMounts:
- mountPath: /config
name: config
ports:
- name: web
containerPort: 80
hostPort: 80
- name: websecure
containerPort: 443
hostPort: 443
- name: admin
containerPort: 8080
- name: mysql
containerPort: 3312
hostPort: 3306
- name: redis
containerPort: 6379
hostPort: 6379
volumes:
- name: config
configMap:
name: traefik
删除原deploy,再重新创建
# kubectl delete -f traefik-deploy.yaml
# kubectl apply -f traefik-deploy.yaml
验证是否添加成功
# kubectl get daemonset traefik -n kube-system -o yaml
...
- containerPort: 6379
hostPort: 6379
name: redis
protocol: TCP
...
# vim redis-ingressroutetcp.yaml
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRouteTCP
metadata:
name: redis
namespace: default
spec:
entryPoints:
- redis
routes:
- match: HostSNI(`*`)
services:
- name: redis
port: 6379
# kubectl get ingressroutetcp
NAME AGE
redis 8s
在集群之外主机添加域名解析
# cat /etc/hosts
192.168.10.15 redis.test.com
# wget http://download.redis.io/releases/redis-3.2.8.tar.gz
# tar xf redis-3.2.8.tar.gz
# make
# ./src/redis-cli -h redis.kubemsb.com -p 6379
redis.kubemsb.com:6379> ping
PONG
redis.kubemsb.com:6379> set hello 123
OK
redis.kubemsb.com:6379> get hello
"123"
3.5 配置UDP路由规则
# vim whoami-ingressrouteudp.yaml
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRouteUDP
metadata:
name: ingressrouteudpwho
spec:
entryPoints:
- udpep
routes:
- services:
- name: whoamiudp
port: 8080
# kubectl get ingressrouteudp
NAME AGE
ingressrouteudpwho 11s
验证可有性
# kubectl get svc
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
whoamiudp ClusterIP 10.96.85.175 8080/TCP 6h35m
echo "WHO" | socat - udp4-datagram:10.96.85.175:8080
echo "othermessage" | socat - udp4-datagram:10.96.85.175:8080
输出
# echo "WHO" | socat - udp4-datagram:192.168.10.12:8084
Hostname: whoamiudp-58f6cf7b8-54552
IP: 127.0.0.1
IP: ::1
IP: 10.244.159.187
IP: fe80::1cb4:e1ff:fe66:d9b
# echo "othermessage" | socat - udp4-datagram:192.168.10.12:8084
Received: othermessage
四、traefik中间件 MiddleWare
4.1 traefik中间件介绍 MiddleWare
中间件是 Traefik2.0 中一个非常有特色的功能，我们可以根据自己的各种需求去选择不同的中间件来满足服务，Traefik 官方已经内置了许多不同功能的中间件，其中包括修改请求头信息；重定向；身份验证等等，而且中间件还可以通过链式组合的方式来适用各种情况。例如：强制跳转https、去除访问前缀、访问白名单等。
4.2 traefik 中间件应用案例 ipWhiteList
在工作 中，有一些URL并不希望对外暴露，比如prometheus、grafana等，我们就可以通过白名单IP来过到要求，可以使用Traefix中的ipWhiteList中间件来完成。
# vim deploy-service.yaml
kind: Deployment
apiVersion: apps/v1
metadata:
name: nginx-web-middleware
namespace: default
spec:
replicas: 1
selector:
matchLabels:
app: middle
template:
metadata:
labels:
app: middle
spec:
containers:
- name: nginx-web-c
image: nginx:latest
ports:
- containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
name: service-middle
namespace: default
spec:
ports:
- name: http
port: 80
selector:
app: middle
# kubectl apply -f deploy-service.yaml
deployment.apps/nginx-web-middleware created
service/service-middle created
# vim middleware-ipwhitelist.yaml
apiVersion: traefik.containo.us/v1alpha1
kind: Middleware
metadata:
name: gs-ipwhitelist
spec:
ipWhiteList:
sourceRange:
- 127.0.0.1
- 10.244.0.0/16
- 10.96.0.0/12
- 192.168.10.0/24
# kubectl apply -f middleware-ipwhitelist.yaml
middleware.traefik.containo.us/gs-ipwhitelist created
# vim deploy-service-middle.yaml
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
name: ingressroutemiddle
namespace: default
spec:
entryPoints:
- web
routes:
- match: Host(`middleware.test.com`) && PathPrefix(`/`)
kind: Rule
services:
- name: service-middle
port: 80
namespace: default
middlewares:
- name: gs-ipwhitelist
# kubectl apply -f deploy-service-middle.yaml
ingressroute.traefik.containo.us/ingressroutemiddle created
在集群之外的主机上访问
# vim /etc/hosts
192.168.10.15 middleware.test.com
# curl http://middleware.test.com
把集群外主机所在的网段从白名单中删除，发现无法访问。
# cat middleware-ipwhitelist.yaml
apiVersion: traefik.containo.us/v1alpha1
kind: Middleware
metadata:
name: gs-ipwhitelist
spec:
ipWhiteList:
sourceRange:
- 127.0.0.1
- 10.244.0.0/16
- 10.96.0.0/12
# kubectl apply -f middleware-ipwhitelist.yaml
middleware.traefik.containo.us/gs-ipwhitelist configured
在集群外主机访问
# curl http://middleware.test.com
Forbidden
五、traefik高级应用
在实际的生产环境，除了上线业务之外，还有更复杂的使用要求。
在开始traefik的高级用法之前，还需要了解一个TraefikService，通过把TraefikService注册到CRD来实现更复杂的请求设置。
TraefikService 目前能用于以下功能
servers load balancing.（负载均衡）
services Weighted Round Robin load balancing.（权重轮询）
services mirroring.（镜像）
5.1 负载均衡
5.1.1 创建deployment控制器类型应用
# vim loadbalancer-deploy.yaml
kind: Deployment
apiVersion: apps/v1
metadata:
name: nginx-web1
namespace: default
spec:
replicas: 1
selector:
matchLabels:
app: v1
template:
metadata:
labels:
app: v1
spec:
containers:
- name: nginx-web-c
image: nginx:latest
ports:
- containerPort: 80
---
kind: Deployment
apiVersion: apps/v1
metadata:
name: nginx-web2
namespace: default
spec:
replicas: 1
selector:
matchLabels:
app: v2
template:
metadata:
labels:
app: v2
spec:
containers:
- name: nginx-web-c
image: nginx:latest
ports:
- containerPort: 80
# kubectl apply -f loadbalancer-deploy.yaml
修改容器中网站页面信息
# kubectl get pods
NAME READY STATUS RESTARTS AGE
nginx-web1-856c759646-58snd 1/1 Running 0 2m34s
nginx-web2-5d547f7c5f-vd8n7 1/1 Running 0 2m34s
# kubectl exec -it nginx-web1-856c759646-58snd -- sh
# echo "svc1" > /usr/share/nginx/html/index.html
# exit
# kubectl exec -it nginx-web2-5d547f7c5f-vd8n7 -- sh
# echo "svc2" > /usr/share/nginx/html/index.html
# exit
# vim loadbalancer-deploy.yaml
kind: Deployment
apiVersion: apps/v1
metadata:
name: nginx-web1
namespace: default
spec:
replicas: 1
selector:
matchLabels:
app: v1
template:
metadata:
labels:
app: v1
spec:
containers:
- name: nginx-web-c
image: nginx:latest
lifecycle:
postStart:
exec:
command: ["/bin/sh", "-c", "echo svc1 > /usr/share/nginx/html/index.html"]
ports:
- containerPort: 80
---
kind: Deployment
apiVersion: apps/v1
metadata:
name: nginx-web2
namespace: default
spec:
replicas: 1
selector:
matchLabels:
app: v2
template:
metadata:
labels:
app: v2
spec:
containers:
- name: nginx-web-c
image: nginx:latest
lifecycle:
postStart:
exec:
command: ["/bin/sh", "-c", "echo svc2 > /usr/share/nginx/html/index.html"]
ports:
- containerPort: 80
5.1.2 创建service
# vim loadbalancer-deploy-service.yaml
apiVersion: v1
kind: Service
metadata:
name: svc1
namespace: default
spec:
ports:
- name: http
port: 80
selector:
app: v1
---
apiVersion: v1
kind: Service
metadata:
name: svc2
namespace: default
spec:
ports:
- name: http
port: 80
selector:
app: v2
# kubectl apply -f loadbalancer-deploy-service.yaml
# kubectl get svc
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
svc1 ClusterIP 10.96.136.245 80/TCP 3s
svc2 ClusterIP 10.96.2.4 80/TCP 3s
# curl http://10.96.136.245
svc1
# curl http://10.96.2.4
svc2
5.1.3 创建ingressroute
# vim loadbalancer-deploy-service-ingressroute.yaml
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
name: ingressrouteweblb
namespace: default
spec:
entryPoints:
- web
routes:
- match: Host(`lb.test.com`) && PathPrefix(`/`)
kind: Rule
services:
- name: svc1
port: 80
namespace: default
- name: svc2
port: 80
namespace: default
# kubectl apply -f loadbalancer-deploy-service-ingressroute.yaml
# kubectl get ingressroute
NAME AGE
ingressrouteweblb 9s
5.1.4 访问验证
在集群之外的主机上访问
# cat /etc/hosts
192.168.10.15 lb.test.com
# curl http://lb.test.com
svc1
# curl http://lb.test.com
svc2
5.2 灰度发布
基于上述负载均衡案例基础之上实施。
灰度发布也称为金丝雀发布，让一部分即将上线的服务发布到线上，观察是否达到上线要求，主要通过权重轮询的方式实现。
5.2.1 创建TraefikService
# vim traefikservice.yaml
apiVersion: traefik.containo.us/v1alpha1
kind: TraefikService
metadata:
name: wrr
namespace: default
spec:
weighted:
services:
- name: svc1
port: 80
weight: 3 # 定义权重
kind: Service # 可选，默认就是 Service
- name: svc2
port: 80
weight: 1
# kubectl apply -f traefikservice.yaml
traefikservice.traefik.containo.us/wrr created
5.2.2 创建ingressroute
需要注意的是现在我们配置的 Service 不再是直接的 Kubernetes 对象了，而是上面我们定义的 TraefikService 对象
# vim traefik-wrr.yaml
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
name: ingressroutewrr
namespace: default
spec:
entryPoints:
- web
routes:
- match: Host(`lb.test.com`) && PathPrefix(`/`)
kind: Rule
services:
- name: wrr
namespace: default
kind: TraefikService
# kubectl apply -f traefik-wrr.yaml
ingressroute.traefik.containo.us/ingressroutewrr created
第一次
# curl http://lb.kubemsb.com
svc1
第二次
# curl http://lb.kubemsb.com
svc1
第三次
# curl http://lb.kubemsb.com
svc2
第四次
# curl http://lb.kubemsb.com
svc1
5.3 流量复制
在负责均衡案例基础之上实施
所谓的流量复制，也称为镜像服务是指将请求的流量按规则复制一份发送给其它服务，并且会忽略这部分请求的响应，这个功能在做一些压测或者问题复现的时候很有用。
5.3.1 指定流量来自己于kubernetes service对象
# vim mirror_from_service.yaml
apiVersion: traefik.containo.us/v1alpha1
kind: TraefikService
metadata:
name: mirror-from-service
namespace: default
spec:
mirroring:
name: svc1 # 发送 100% 的请求到 K8S 的 Service "v1"
port: 80
mirrors:
- name: svc2 # 然后复制 20% 的请求到 v2
port: 80
percent: 20
# kubectl apply -f mirror_from_service.yaml
traefikservice.traefik.containo.us/mirror-from-service created
# vim mirror-from-service-ingressroute.yaml
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
name: ingressroute-mirror
namespace: default
spec:
entryPoints:
- web
routes:
- match: Host(`lb.test.com`) && PathPrefix(`/`)
kind: Rule
services:
- name: mirror-from-service
namespace: default
kind: TraefikService
# kubectl apply -f mirror-from-service-ingressroute.yaml
ingressroute.traefik.containo.us/ingressroute-mirror created
# while true
do
curl http://lb.test.com
sleep 1
done
5.3.2 通过traefikservice导入流量
# vim mirror-from-traefikservice.yaml
apiVersion: traefik.containo.us/v1alpha1
kind: TraefikService
metadata:
name: mirror-from-traefikservice
namespace: default
spec:
mirroring:
name: mirror-from-service #流量入口从TraefikService 来
kind: TraefikService
mirrors:
- name: svc2
port: 80
percent: 20
# kubectl apply -f mirror-from-traefikservice.yaml
traefikservice.traefik.containo.us/mirror-from-traefikservice created
# vim mirror-from-traefikservice-ingressroute.yaml
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
name: ingressroute-mirror-traefikservice
namespace: default
spec:
entryPoints:
- web
routes:
- match: Host(`lb.test.com`) && PathPrefix(`/`)
kind: Rule
services:
- name: mirror-from-traefikservice
namespace: default
kind: TraefikService
# kubectl apply -f mirror-from-tradfikservice-ingressroute.yaml
ingressroute.traefik.containo.us/ingressroute-mirror-traefikservice created
# while true; do curl http://lb.test.com; sleep 1; done
5.3.3 流量复制小结
通过上述的演示我们会发现所有的流量100%发送了svc1,有20%的流量被复制到svc2，且用户收到响应均来自svc1，svc2并没有响应，可通过查看svc1及svc2应用日志获取访问日志。
六、Kubernetes Gateway API
6.1 Gateway API介绍
6.1.1 Gateway API架构
Gateway API（之前叫 Service API）是由 SIG-NETWORK 社区管理的开源项目，项目地址：https://gateway-api.sigs.k8s.io/。主要原因是 Ingress 资源对象不能很好的满足网络需求，很多场景下 Ingress 控制器都需要通过定义 annotations 或者 crd 来进行功能扩展，这对于使用标准和支持是非常不利的，新推出的 Gateway API 旨在通过可扩展的面向角色的接口来增强服务网络。
Gateway API 是 Kubernetes 中的一个 API 资源集合，包括 GatewayClass、Gateway、HTTPRoute、TCPRoute、Service 等，这些资源共同为各种网络用例构建模型。
Gateway API 的改进比当前的 Ingress 资源对象有很多更好的设计：
- 面向角色 - Gateway 由各种 API 资源组成，这些资源根据使用和配置 Kubernetes 服务网络的角色进行建模。
- 通用性 - 和 Ingress 一样是一个具有众多实现的通用规范，Gateway API 是一个被设计成由许多实现支持的规范标准。
- 更具表现力 - Gateway API 资源支持基于 Header 头的匹配、流量权重等核心功能，这些功能在 Ingress 中只能通过自定义注解才能实现。
- 可扩展性 - Gateway API 允许自定义资源链接到 API 的各个层，这就允许在 API 结构的适当位置进行更精细的定制。
还有一些其他值得关注的功能：
- GatewayClasses - GatewayClasses 将负载均衡实现的类型形式化，这些类使用户可以很容易了解到通过 Kubernetes 资源可以获得什么样的能力。
- 共享网关和跨命名空间支持 - 它们允许共享负载均衡器和 VIP，允许独立的路由资源绑定到同一个网关，这使得团队可以安全地共享（包括跨命名空间）基础设施，而不需要直接协调。
- 规范化路由和后端 - Gateway API 支持类型化的路由资源和不同类型的后端，这使得 API 可以灵活地支持各种协议（如 HTTP 和 gRPC）和各种后端服务（如 Kubernetes Service、存储桶或函数）。
6.1.2 面向角色设计
无论是道路、电力、数据中心还是 Kubernetes 集群，基础设施都是为了共享而建的，然而共享基础设施提供了一个共同的挑战，那就是如何为基础设施用户提供灵活性的同时还能被所有者控制。
Gateway API 通过对 Kubernetes 服务网络进行面向角色的设计来实现这一目标，平衡了灵活性和集中控制。它允许共享的网络基础设施（硬件负载均衡器、云网络、集群托管的代理等）被许多不同的团队使用，所有这些都受到集群运维设置的各种策略和约束。下面的例子显示了是如何在实践中运行的。
一个集群运维人员创建了一个基于 GatewayClass 的 Gateway 资源，这个 Gateway 部署或配置了它所代表的基础网络资源，集群运维和特定的团队必须沟通什么可以附加到这个 Gateway 上来暴露他们的应用。集中的策略，如 TLS，可以由集群运维在 Gateway 上强制执行，同时，Store 和 Site 应用在他们自己的命名空间中运行，但将他们的路由附加到相同的共享网关上，允许他们独立控制他们的路由逻辑。
这种关注点分离的设计可以使不同的团队能够管理他们自己的流量，同时将集中的策略和控制留给集群运维。
6.1.3 Gateway API概念
在整个 Gateway API 中涉及到3个角色：基础设施提供商、集群管理员、应用开发人员，在某些场景下可能还会涉及到应用管理员等角色。Gateway API 中定义了3种主要的资源模型：GatewayClass、Gateway、Route。
6.1.3.1 GatewayClass
GatewayClass 定义了一组共享相同配置和动作的网关。每个GatewayClass 由一个控制器处理，是一个集群范围的资源，必须至少有一个 GatewayClass 被定义。
这与 Ingress 的 IngressClass 类似，在 Ingress v1beta1 版本中，与 GatewayClass 类似的是 ingress-class 注解，而在Ingress V1 版本中，最接近的就是 IngressClass 资源对象。
6.1.3.2 Gateway
Gateway 网关描述了如何将流量转化为集群内的服务，也就是说，它定义了一个请求，要求将流量从不了解 Kubernetes 的地方转换到集群内的服务。例如，由云端负载均衡器、集群内代理或外部硬件负载均衡器发送到 Kubernetes 服务的流量。
它定义了对特定负载均衡器配置的请求，该配置实现了 GatewayClass 的配置和行为规范，该资源可以由管理员直接创建，也可以由处理 GatewayClass 的控制器创建。
Gateway 可以附加到一个或多个路由引用上，这些路由引用的作用是将流量的一个子集导向特定的服务。
6.1.3.3 Route 资源
路由资源定义了特定的规则，用于将请求从网关映射到 Kubernetes 服务。
从 v1alpha2 版本开始，API 中包含四种 Route 路由资源类型，对于其他未定义的协议，鼓励采用特定实现的自定义路由类型，当然未来也可能会添加新的路由类型。
6.1.3.3.1 HTTPRoute
HTTPRoute 适用于 HTTP 或 HTTPS 连接，适用于我们想要检查 HTTP 请求并使用 HTTP 请求进行路由或修改的场景，比如使用 HTTP Headers 头进行路由，或在请求过程中对它们进行修改。
6.1.3.3.2 TLSRoute
TLSRoute 用于 TLS 连接，通过 SNI 进行区分，它适用于希望使用 SNI 作为主要路由方法的地方，并且对 HTTP 等更高级别协议的属性不感兴趣，连接的字节流不经任何检查就被代理到后端。
6.1.3.3.3 TCPRoute 和 UDPRoute
TCPRoute（和UDPRoute）旨在用于将一个或多个端口映射到单个后端。在这种情况下，没有可以用来选择同一端口的不同后端的判别器，所以每个 TCPRoute 在监听器上需要一个不同的端口。你可以使用 TLS，在这种情况下，未加密的字节流会被传递到后端，当然也可以不使用 TLS，这样加密的字节流将传递到后端。
6.1.4 Gateway API概念之间关系
GatewayClass、Gateway、xRoute 和 Service 的组合定义了一个可实施的负载均衡器。下图说明了不同资源之间的关系:
使用反向代理实现的网关的典型客户端/网关 API 请求流程如下所示：
- 客户端向 http://foo.example.com 发出请求
- DNS 将域名解析为 Gateway 网关地址
- 反向代理在监听器上接收请求，并使用 Host Header 来匹配HTTPRoute
- (可选)反向代理可以根据 HTTPRoute 的匹配规则进行路由
- (可选)反向代理可以根据 HTTPRoute 的过滤规则修改请求，即添加或删除 headers
- 最后，反向代理根据 HTTPRoute 的 forwardTo 规则，将请求转发给集群中的一个或多个对象，即服务。
6.2 kubernetes gateway CRD安装
要在 Traefik 中使用 Gateway API，首先我们需要先手动安装 Gateway API 的 CRDs，使用如下命令即可安装，这将安装包括 GatewayClass、Gateway、HTTPRoute、TCPRoute 等 CRDs：
# kubectl apply -k "github.com/kubernetes-sigs/service-apis/config/crd?ref=v0.3.0"
输出：
customresourcedefinition.apiextensions.k8s.io/gatewayclasses.gateway.networking.k8s.io created
customresourcedefinition.apiextensions.k8s.io/gateways.gateway.networking.k8s.io created
customresourcedefinition.apiextensions.k8s.io/httproutes.gateway.networking.k8s.io created
customresourcedefinition.apiextensions.k8s.io/referencepolicies.gateway.networking.k8s.io created
customresourcedefinition.apiextensions.k8s.io/tcproutes.gateway.networking.k8s.io created
customresourcedefinition.apiextensions.k8s.io/tlsroutes.gateway.networking.k8s.io created
customresourcedefinition.apiextensions.k8s.io/udproutes.gateway.networking.k8s.io created
6.3 为traefik授权（RBAC）
# vim traefik-gatewayapi-rbac.yaml
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: gateway-role
rules:
- apiGroups:
- ""
resources:
- services
- endpoints
- secrets
verbs:
- get
- list
- watch
- apiGroups:
- networking.x-k8s.io
resources:
- gatewayclasses
- gateways
- httproutes
- tcproutes
- tlsroutes
verbs:
- get
- list
- watch
- apiGroups:
- networking.x-k8s.io
resources:
- gatewayclasses/status
- gateways/status
- httproutes/status
- tcproutes/status
- tlsroutes/status
verbs:
- update
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: gateway-controller
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: gateway-role
subjects:
- kind: ServiceAccount
name: traefik-ingress-controller
namespace: kube-system
6.4 Traefik开启Gateway api支持
# vim traefik-configmap.yaml
......
providers:
kubernetesCRD: ""
kubernetesingress: ""
kubernetesGateway: "" 添加此行
experimental: 添加此行
kubernetesGateway: true 添加此行
......
删除
# kubectl delete -f traefik-configmap.yaml
# kubectl delete -f traefik-deploy.yaml
重新运行
# kubectl apply -f traefik-configmap.yaml
# kubectl apply -f traefik-deploy.yaml
6.5 创建Gateway API的GatewayClass
# vim gatewayclass.yaml
apiVersion: networking.x-k8s.io/v1alpha1
kind: GatewayClass
metadata:
name: traefik
spec:
controller: traefik.io/gateway-controller
# kubectl apply -f gatewayclass.yaml
# kubectl get gatewayclass
NAME CONTROLLER AGE
traefik traefik.io/gateway-controller 2m59s
6.6 Gateway API应用案例
6.6.1 通过Gateway API方式暴露traefik dashboard
6.6.1.1 创建gateway
# vim gateway.yaml
apiVersion: networking.x-k8s.io/v1alpha1
kind: Gateway
metadata:
name: http-gateway
namespace: kube-system
spec:
gatewayClassName: traefik
listeners:
- protocol: HTTP
port: 80
routes:
kind: HTTPRoute
namespaces:
from: All
selector:
matchLabels:
app: traefik
# kubectl apply -f gateway.yaml
gateway.networking.x-k8s.io/http-gateway created
# kubectl get gateway
NAME CLASS AGE
http-gateway traefik 6s
6.5.1.2 创建HTTPRoute
# vim httproute.yaml
apiVersion: networking.x-k8s.io/v1alpha1
kind: HTTPRoute
metadata:
name: traefix-dashboard-gateway-api-route
namespace: kube-system
labels:
app: traefik
spec:
hostnames:
- "traefikdashboard.test.com"
rules:
- matches:
- path:
type: Prefix
value: /
forwardTo:
- serviceName: traefik
port: 8080
weight: 1
# kubectl apply -f httproute.yaml
httproute.networking.x-k8s.io/traefix-dashboard-gateway-api-route created
# kubectl get httproute
NAME HOSTNAMES AGE
traefix-dashboard-gateway-api-route ["traefikdashboard.test.com"] 6s
6.5.1.3 在集群之外主机访问
# vim /etc/hosts
192.168.10.12 traefikdashboard.test.com
6.6.2 通过Gateway API方式暴露WEB应用
# cat gatewayapi-web.yaml
kind: Deployment
apiVersion: apps/v1
metadata:
name: nginx-web-gatewayapi
namespace: default
spec:
replicas: 1
selector:
matchLabels:
app: gatewayweb
template:
metadata:
labels:
app: gatewayweb
spec:
containers:
- name: nginx-web
image: nginx:latest
lifecycle:
postStart:
exec:
command: ["/bin/sh", "-c", "echo gatewayweb > /usr/share/nginx/html/index.html"]
ports:
- containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
name: nginx-web-gatewayapi-svc
namespace: default
spec:
ports:
- name: http
port: 80
selector:
app: gatewayweb
# kubectl apply -f gatewayapi-web.yaml
deployment.apps/nginx-web-gatewayapi created
service/nginx-web-gatewayapi-svc created
# vim gatewayapi-web-gateway.yaml
apiVersion: networking.x-k8s.io/v1alpha1
kind: Gateway
metadata:
name: nginx-web-gateway
spec:
gatewayClassName: traefik
listeners:
- protocol: HTTP
port: 80
routes:
kind: HTTPRoute
namespaces:
from: All
selector:
matchLabels:
app: gatewayweb
# kubectl apply -f gatewayapi-web-gateway.yaml
gateway.networking.x-k8s.io/nginx-web-gateway created
# kubectl get gateway
NAME CLASS AGE
nginx-web-gateway traefik 7s
# kubectl describe gateway nginx-web-gateway
# cat gatewayapi-web-httproute.yaml
apiVersion: networking.x-k8s.io/v1alpha1
kind: HTTPRoute
metadata:
name: nginx-web-gateway-api-route
labels:
app: gatewayweb
spec:
hostnames:
- "nginx.test.com"
rules:
- matches:
- path:
type: Prefix
value: /
forwardTo:
- serviceName: nginx-web-gatewayapi-svc
port: 80
weight: 1
# kubectl apply -f gatewayapi-web-httproute.yaml
httproute.networking.x-k8s.io/nginx-web-gateway-api-route created
kubectl get httproute
NAME HOSTNAMES AGE
nginx-web-gateway-api-route ["nginx.test.com"] 6s
在集群之外主机访问
# vim /etc/hosts
192.168.10.12 nginx.test.com
# curl http://nginx.kubemsb.com
gatewayweb
6.6.3 金丝雀发布
Gateway APIs 规范可以支持的另一个功能是金丝雀发布，假设你想在一个端点上运行两个不同的服务（或同一服务的两个版本），并将一部分请求路由到每个端点，则可以通过修改你的 HTTPRoute 来实现。
# vim gateway-cn-deploy.yaml
kind: Deployment
apiVersion: apps/v1
metadata:
name: nginx-web1
namespace: kube-system
spec:
replicas: 1
selector:
matchLabels:
app: v1
template:
metadata:
labels:
app: v1
spec:
containers:
- name: nginx-web-c
image: nginx:latest
lifecycle:
postStart:
exec:
command: ["/bin/sh", "-c", "echo svc1 > /usr/share/nginx/html/index.html"]
ports:
- containerPort: 80
---
kind: Deployment
apiVersion: apps/v1
metadata:
name: nginx-web2
namespace: kube-system
spec:
replicas: 1
selector:
matchLabels:
app: v2
template:
metadata:
labels:
app: v2
spec:
containers:
- name: nginx-web-c
image: nginx:latest
lifecycle:
postStart:
exec:
command: ["/bin/sh", "-c", "echo svc2 > /usr/share/nginx/html/index.html"]
ports:
- containerPort: 80
# kubectl apply -f gateway-cn-deploy.yaml
# vim gateway-cn-deploy-svc.yaml
apiVersion: v1
kind: Service
metadata:
name: svc1
namespace: kube-system
spec:
ports:
- name: http
port: 80
selector:
app: v1
---
apiVersion: v1
kind: Service
metadata:
name: svc2
namespace: kube-system
spec:
ports:
- name: http
port: 80
selector:
app: v2
# kubectl apply -fgateway-cn-deploy-svc.yaml
# vim gateway-cn-httproute.yaml
apiVersion: networking.x-k8s.io/v1alpha1
kind: HTTPRoute
metadata:
labels:
app: traefik
name: nginxweb-app
namespace: kube-system
spec:
hostnames:
- canary.test.com
rules:
- forwardTo:
- port: 80
serviceName: svc1
weight: 3 # 3/4 的请求到svc1
- port: 80
serviceName: svc2
weight: 1 # 1/4 的请求到svc2
# kubectl apply -f gateway-cn-httproute.yaml
httproute.networking.x-k8s.io/nginxweb-app created
# kubectl get httproute -n kube-system
NAME HOSTNAMES AGE
nginxweb-app ["canary.test.com"] 7s
在 集群之外主机访问
# vim /etc/hosts
192.168.10.12 canary.test.com
[root@dockerhost ~]# curl http://canary.test.com
svc1
[root@dockerhost ~]# curl http://canary.test.com
svc1
[root@dockerhost ~]# curl http://canary.test.com
svc1
[root@dockerhost ~]# curl http://canary.test.com
svc2
以上使用 Traefik 来测试了 Kubernetes Gateway APIs 的使用。目前，Traefik 对 Gateway APIs 的实现是基于 v1alpha1 版本的规范，目前最新的规范是 v1alpha2，所以和最新的规范可能有一些出入的地方。