目标：
部署三个服务traefik-ui，grafana，prometheus，并通过traefik 反向代理。
| service | namespaces | domain name | https |
|---|---|---|---|
| traefik-ui | traefik | traefik.qyd.com | Y |
| grafana | kube-system | grafana.dfb.com | N |
| prometheus | kube-system | prometheus.qyd.com | Y |
步骤：
1、部署traefik
相关资源yml
- https://github.com/xiaocaisgit/k8sAbout/blob/master/traefik/rbac.yml
- https://github.com/xiaocaisgit/k8sAbout/blob/master/traefik/deployment.yml
- https://github.com/xiaocaisgit/k8sAbout/blob/master/traefik/configmap.yml
- https://github.com/xiaocaisgit/k8sAbout/blob/master/traefik/prometheus-ingress.yml
- https://github.com/xiaocaisgit/k8sAbout/blob/master/traefik/grafana-ingress.yml
- https://github.com/xiaocaisgit/k8sAbout/blob/master/traefik/traefik-web-ui.yml
创建traefik 这个命名空间，使用configmap 挂载配置。
kubectl create cm -n traefik traefik-config --from-file=traefik.tomlapiVersion: v1
items:
- apiVersion: v1
data:
traefik.toml: |
graceTimeOut = 10
traefikLogsFile = "/log/traefik.log"
accessLogsFile = "/log/access.log"
logLevel = "INFO"
MaxIdleConnsPerHost = 60
InsecureSkipVerify = true
defaultEntryPoints = ["https","http"]
[entryPoints]
[entryPoints.http]
address = ":80"
[entryPoints.http.redirect]
regex = "^http://(.*).qyd.com/(.*)"
replacement = "https://$1.qyd.com/$2"
[entryPoints.https]
address = ":443"
[entryPoints.https.tls]
[[entryPoints.https.tls.certificates]]
certFile = "/ssl/qyd/tls.crt"
keyFile = "/ssl/qyd/tls.key"
[[entryPoints.https.tls.certificates]]
certFile = "/ssl/dfb/tls.crt"
keyFile = "/ssl/dfb/tls.key"
[metrics]
[metrics.prometheus]
entryPoint = "traefik"
kind: ConfigMap
metadata:
name: traefik-config
namespace: traefik
kind: List
metadata:
resourceVersion: ""
selfLink: ""
获取 qyd.com 和dfb.com 两个域名的证书，并创建secret。
kubectl create secret generic dfb-tls-cert --from-file=dfb/tls.crt --from-file=dfb/tls.key -n traefik
kubectl create secret generic qyd-tls-cert --from-file=qyd/tls.crt --from-file=qyd/tls.key -n traefik部署traefik-ingreess-controller
kubectl app -f rbac.yml---
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
name: traefik-ingress-controller
rules:
- apiGroups:
- ""
resources:
- services
- endpoints
- secrets
verbs:
- get
- list
- watch
- apiGroups:
- extensions
resources:
- ingresses
verbs:
- get
- list
- watch
- apiGroups:
- extensions
resources:
- ingresses/status
verbs:
- update
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
name: traefik-ingress-controller
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: traefik-ingress-controller
subjects:
- kind: ServiceAccount
name: traefik-ingress-controller
namespace: traefik
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: traefik-ingress-controller
namespace: traefik
kubectl apply -f deployment.yml apiVersion: extensions/v1beta1
kind: Deployment
metadata:
labels:
k8s-app: traefik-ingress-lb
name: traefik-ingress-controller
namespace: traefik
spec:
replicas: 1
selector:
matchLabels:
k8s-app: traefik-ingress-lb
strategy:
rollingUpdate:
maxSurge: 1
maxUnavailable: 1
type: RollingUpdate
template:
metadata:
labels:
k8s-app: traefik-ingress-lb
name: traefik-ingress-lb
spec:
containers:
- args:
- --configFile=/etc/traefik/traefik.yml
- --api
- --kubernetes
image: itanony.com/repository/docker-hosted/test/treafik:v1.7.10
imagePullPolicy: IfNotPresent
name: traefik-ingress-lb
ports:
- containerPort: 80
hostPort: 80
name: http
protocol: TCP
- containerPort: 8080
hostPort: 8080
name: admin
protocol: TCP
- containerPort: 443
hostPort: 443
name: https
protocol: TCP
resources: {}
terminationMessagePath: /dev/termination-log
terminationMessagePolicy: File
volumeMounts:
- mountPath: /etc/traefik/
name: config
- mountPath: /ssl/qyd/
name: qyd-cert
- mountPath: /ssl/dfb/
name: dfb-cert
- mountPath: /log/
name: logs
dnsPolicy: ClusterFirst
hostNetwork: true
nodeSelector:
cpu: high
restartPolicy: Always
schedulerName: default-scheduler
securityContext: {}
serviceAccount: traefik-ingress-controller
serviceAccountName: traefik-ingress-controller
terminationGracePeriodSeconds: 60
volumes:
- name: qyd-cert
secret:
defaultMode: 420
secretName: qyd-tls-cert
- name: dfb-cert
secret:
defaultMode: 420
secretName: dfb-tls-cert
- configMap:
defaultMode: 420
name: traefik-config
name: config
- hostPath:
path: /var/log/traefik
type: ""
name: logs
注意deployment.yml 中修改images地址。另外因为是测试，故采用nodeselector 只部署到一台固定的node节点，采用宿主机网络模式。ingress controller 的高可用留在以后研究。
查看pod 状态
kubectl get pods -n traefiktraefik 启动后会监控一个8080 的端口提供一个管理的web-ui，可以查看frontend 和backend 的对应关系，及一些基本的监控数据
我们创建一个ClusterIP 的service，并创建ingress，通过traefik 使用traefik.qyd.com 域名来反向代理
kubectl apply -f traefik-web-ui.ymlapiVersion: v1
kind: Service
metadata:
name: traefik-web-ui
namespace: traefik
spec:
selector:
k8s-app: traefik-ingress-lb
ports:
- name: web
port: 80
targetPort: 8080
---
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: traefik-web-ui
namespace: traefik
spec:
rules:
- host: traefik.qyd.com
http:
paths:
- path: /
backend:
serviceName: traefik-web-ui
servicePort: web在本机hosts中添加 traefik.qyd.com 的hosts 记录解析到traefik 部署的node节点。
通过浏览器访问。页面正常显示，并且使用http 访问时会自动跳转到https。
部署prometheus 和grafana 代理
这里只讨论通过traefik-ingres 代理prometheus 和grafan。部署过程请Google。
创建prometheus 和 grafana 的ingress 。 通过traefik 分别使用 prometheus.yd.com 和grafana.dfb.com 反向代理。
注意yml 中namespace，serviceName，servicePort 与自己集群中服务的名称一致。
kubectl apply -f grafana-ingress.yml
kubectl apply -f prometheus-ingress.ymlapiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: grafana
namespace: kube-system
spec:
rules:
- host: grafana.dfb.com
http:
paths:
- backend:
serviceName: monitoring-grafana
servicePort: 80
path: /
---
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: prometheus
namespace: kube-system
spec:
rules:
- host: prometheus.qyd.com
http:
paths:
- backend:
serviceName: prometheus
servicePort: prometheus
path: /同样在本机hosts 中添加两个域名的解析记录。通过浏览器访问正常，prometheus.qyd.com访问http 会rewrite到https，grafana.dfb.com不会做rewrite。至此部署部分结束
配置解析
多域名 配置https，我们不需要对每一个域名指定证书， 只需要在entrypoints 中指定证书路径。traefik 会自动根据请求中的主机头和证书中的CN进行匹配。
生产中可能遇到同一个反向代理下。 有的域名需要启用https 的强制rewrite。 有些则不能做强制rewrite。traefik 提供entryPoints.http.redirect 通过正则来对需要rewrite 的域名进行正则匹配。 这里感觉有点不灵活。 也可能还有更好的方式。