之前文章介绍了常用一些Ingress Controller,如下：
- HAProxy Ingress Controller
- Nginx Ingress Controller
- Traefik Ingress Controller
- Kong Ingress Controller
今天来详细介绍两种常用的Ingress Controller.
二、Nginx Ingress
2.1、安装
2.1.1 在线安装
在线安装直接执行以下命令：
kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/static/mandatory.yaml
然后安装NodePort：
kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/static/provider/baremetal/service-nodeport.yaml
2.1.2、离线安装
到https://github.com/kubernetes/ingress-nginx/tree/master/deploy/static这个下面下载对应的YAML文件，有configmap.yaml，namespace.yaml，rbac.yaml，with-rbac.yaml，可以写一个如下循环下载：
for yaml in configmap.yaml namespace.yaml rbac.yaml with-rbac.yaml; do wget https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/static/$yaml;done
然后执行如下命令安装，前提先执行namespace.yaml这个YAML文件：
kubectl apply -f namespace.yaml
kubectl apply -f .
然后我们可以执行以下命令查看：
[root@master nginx]# kubectl get pod -n ingress-nginx
NAME READY STATUS RESTARTS AGE
nginx-ingress-controller-799dbf6fbd-zxvmp 0/1 ContainerCreating 0 12s
然后安装nodePort:
wget https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/static/provider/baremetal/service-nodeport.yaml
kubectl apply -f service-nodeport.yaml
然后我修改了nodePort的端口，如下：
apiVersion: v1
kind: Service
metadata:
name: ingress-nginx
namespace: ingress-nginx
labels:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
spec:
type: NodePort
ports:
- name: http
port: 80
targetPort: 80
nodePort: 30080
protocol: TCP
- name: https
port: 443
targetPort: 443
nodePort: 30443
protocol: TCP
selector:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
---
然后查看结果：
[root@master nginx]# kubectl get pod -n ingress-nginx
NAME READY STATUS RESTARTS AGE
nginx-ingress-controller-799dbf6fbd-zxvmp 1/1 Running 0 52m
[root@master nginx]# kubectl get svc -n ingress-nginx
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
ingress-nginx NodePort 10.68.194.177 80:30080/TCP,443:30443/TCP 46m
2.2、使用
先创建Pod，service，定义如下YAML：
[root@master test]# cat nginx-ingress-demo.yaml
---
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: nginx-deploy
namespace: default
spec:
replicas: 3
selector:
matchLabels:
app: nginx-demo
release: canary
template:
metadata:
name: my-nginx
labels:
app: nginx-demo
release: canary
spec:
containers:
- name: my-nginx
image: nginx
imagePullPolicy: IfNotPresent
ports:
- name: http
containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
name: nginx-svc
namespace: default
spec:
selector:
app: nginx-demo
release: canary
ports:
- name: http
port: 80
targetPort: 80
然后执行kubectl apply -f nginx-ingress-demo.yaml，查看结果：
[root@master test]# kubectl get pod
NAME READY STATUS RESTARTS AGE
nginx-deploy-549cb5c7ff-65fbb 1/1 Running 0 21s
nginx-deploy-549cb5c7ff-75fmt 1/1 Running 0 21s
nginx-deploy-549cb5c7ff-wg4w7 1/1 Running 0 17s
[root@master test]# kubectl get svc
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
kubernetes ClusterIP 10.68.0.1 443/TCP 5d2h
nginx-svc ClusterIP 10.68.214.28 80/TCP 6m21s
然后定义Ingress Nginx，YAML文件如下：
[root@master test]# cat ingress-nginx.yaml
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
name: ingress-nginx
annotations:
kubernetes.io/ingress.class: "nginx"
spec:
rules:
- host:
http:
paths:
- path:
backend:
serviceName: nginx-svc
servicePort: 80
然后查看结果：
[root@master test]# kubectl describe ingresses ingress-nginx
Name: ingress-nginx
Namespace: default
Address: 10.68.194.177
Default backend: default-http-backend:80 ()
Rules:
Host Path Backends
---- ---- --------
*
nginx-svc:80 (172.20.1.61:80,172.20.1.62:80,172.20.2.87:80)
Annotations:
kubectl.kubernetes.io/last-applied-configuration: {"apiVersion":"networking.k8s.io/v1beta1","kind":"Ingress","metadata":{"annotations":{"kubernetes.io/ingress.class":"nginx"},"name":"ingress-nginx","namespace":"default"},"spec":{"rules":[{"host":null,"http":{"paths":[{"backend":{"serviceName":"nginx-svc","servicePort":80},"path":null}]}}]}}
kubernetes.io/ingress.class: nginx
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal CREATE 11m nginx-ingress-controller Ingress default/ingress-nginx
Normal UPDATE 77s (x3 over 11m) nginx-ingress-controller Ingress default/ingress-nginx
在流量器访问：
image.png
2.3、TLS
Nginx Ingress不止支持HTTP，还支持HTTPS。我们使用如下命令手动创建证书：
openssl genrsa -out tls.key 2048
openssl req -new -x509 -key tls.key -out tls.crt -subj "/C=CN/ST=Chongqing/L=Chongqing/OU=DevOps/CN=nginx.joker.com"
然后配置Secrect：
kubectl create secret tls tls-secret --key tls.key --cert tls.crt
然后查看：
[root@master test]# kubectl get secret
NAME TYPE DATA AGE
default-token-gmdbb kubernetes.io/service-account-token 3 5d3h
tls-secret kubernetes.io/tls 2 20s
[root@master test]# kubectl describe secrets tls-secret
Name: tls-secret
Namespace: default
Labels:
Annotations:
Type: kubernetes.io/tls
Data
====
tls.crt: 1302 bytes
tls.key: 1679 bytes
配置Ingress nginx:
[root@master test]# cat ingress-nginx-tls.yaml
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
name: ingress-nginx-tls
annotations:
kubernetes.io/ingress.class: "nginx"
spec:
tls:
- host:
secretName: tls-secret
rules:
- host:
http:
paths:
- path:
backend:
serviceName: nginx-svc
servicePort: 80
然后查看结果：
[root@master test]# kubectl get ingresses.
NAME HOSTS ADDRESS PORTS AGE
ingress-nginx-tls * 80, 443 4s
浏览器访问：
image.png
三、Traefik Ingress
3.1、安装
创建namespace（namespace.yaml）：
apiVersion: v1
kind: Namespace
metadata:
name: ingress-traefik
labels:
app.kubernetes.io/name: ingress-traefik
app.kubernetes.io/part-of: ingress-traefik
---
创建RBAC安全认证（rdac.yaml）：
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: traefik-ingress-controller
namespace: ingress-traefik
---
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
name: traefik-ingress-controller
rules:
- apiGroups:
- ""
resources:
- services
- endpoints
- secrets
verbs:
- get
- list
- watch
- apiGroups:
- extensions
resources:
- ingresses
verbs:
- get
- list
- watch
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
name: traefik-ingress-controller
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: traefik-ingress-controller
subjects:
- kind: ServiceAccount
name: traefik-ingress-controller
namespace: ingress-traefik
配置traefik（traefik.yaml）：
---
kind: Deployment
apiVersion: extensions/v1beta1
metadata:
name: traefik-ingress-controller
namespace: ingress-traefik
labels:
k8s-app: traefik-ingress-lb
spec:
replicas: 1
selector:
matchLabels:
k8s-app: traefik-ingress-lb
template:
metadata:
labels:
k8s-app: traefik-ingress-lb
name: traefik-ingress-lb
spec:
serviceAccountName: traefik-ingress-controller
terminationGracePeriodSeconds: 60
# tolerations:
# - operator: "Exists"
# nodeSelector:
# kubernetes.io/hostname: master
containers:
- image: traefik:v1.7.17
name: traefik-ingress-lb
ports:
- name: http
containerPort: 80
- name: admin
containerPort: 8080
args:
- --api
- --kubernetes
- --logLevel=INFO
---
kind: Service
apiVersion: v1
metadata:
name: traefik-ingress-service
namespace: ingress-traefik
spec:
selector:
k8s-app: traefik-ingress-lb
ports:
- protocol: TCP
port: 80
name: web
nodePort: 38000
- protocol: TCP
port: 8080
nodePort: 38080
name: admin
type: NodePort
然后执行如下命令创建：
kubectl apply -f .
查看其结果：
[root@master traefik]# kubectl get svc -n ingress-traefik
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
traefik-ingress-service NodePort 10.68.173.196 80:38000/TCP,8080:38080/TCP 9m19s
[root@master traefik]# kubectl get pod -n ingress-traefik
NAME READY STATUS RESTARTS AGE
traefik-ingress-controller-86769d5d99-fvf82 1/1 Running 0 7m10s
Traefik提供一个WEB UI工具，就是上面8080端口，可以通过其映射的NodePort端口在浏览器访问，如下：
image.png
3.2、使用
使用和上面nginx ingress一样，只是在annotations里配置的kubernetes.io/ingress.class: "traefik"。
3.3、配置
生成SSL证书：
openssl req -newkey rsa:2048 -nodes -keyout tls.key -x509 -days 365 -out tls.crt
创建Secret来存储证书：
# kubectl create secret generic traefik-cert --from-file=tls.crt --from-file=tls.key -n ingress-traefik
配置traefik，让其支持https，配置文件如下：
defaultEntryPoints = ["http", "https"]
[entryPoints]
[entryPoints.http]
address = ":80"
[entryPoints.http.redirect]
entryPoint = "https"
[entryPoints.https]
address = ":443"
[entryPoints.https.tls]
[[entryPoints.https.tls.certificates]]
CertFile = "/ssl/tls.crt"
KeyFile = "/ssl/tls.key"
将上面的 traefik.toml 配置文件通过一个 ConfigMap 对象挂载到 traefik pod 中去。
创建configMap：
# kubectl create configmap traefik-conf --from-file=traefik.toml -n ingress-traefik
更改traefik的YAML文件：
---
kind: Deployment
apiVersion: extensions/v1beta1
metadata:
name: traefik-ingress-controller
namespace: ingress-traefik
labels:
k8s-app: traefik-ingress-lb
spec:
replicas: 1
selector:
matchLabels:
k8s-app: traefik-ingress-lb
name: traefik-ingress-lb
template:
metadata:
labels:
k8s-app: traefik-ingress-lb
name: traefik-ingress-lb
spec:
serviceAccountName: traefik-ingress-controller
terminationGracePeriodSeconds: 60
# tolerations:
# - operator: "Exists"
# nodeSelector:
# kubernetes.io/hostname: master
containers:
- image: traefik:v1.7.17
name: traefik-ingress-lb
volumeMounts:
- name: ssl
mountPath: "/ssl"
- name: config
mountPath: "/config"
ports:
- name: http
containerPort: 80
- name: https
containerPort: 443
- name: admin
containerPort: 8080
args:
- --configfile=/config/traefik.toml
- --api
- --kubernetes
- --logLevel=INFO
volumes:
- name: ssl
secret:
secretName: traefik-cert
- name: config
configMap:
name: traefik-conf
---
kind: Service
apiVersion: v1
metadata:
name: traefik-ingress-service
namespace: ingress-traefik
spec:
selector:
k8s-app: traefik-ingress-lb
ports:
- protocol: TCP
port: 80
name: web
nodePort: 38000
- protocol: TCP
port: 8080
nodePort: 38080
name: admin
type: NodePort
然后生成配置清单，并查看日志观察其是否生效；
# kubectl apply -f traefik.yaml
# kubectl logs traefik-ingress-controller-867f5668dd-7p7tv -n ingress-traefik
time="2019-11-25T08:25:30Z" level=info msg="Using TOML configuration file /config/traefik.toml"
time="2019-11-25T08:25:30Z" level=info msg="No tls.defaultCertificate given for https: using the first item in tls.certificates as a fallback."
time="2019-11-25T08:25:30Z" level=info msg="Traefik version v1.7.17 built on 2019-09-23_06:22:08PM"
time="2019-11-25T08:25:30Z" level=info msg="\nStats collection is disabled.\nHelp us improve Traefik by turning this feature on :)\nMore details on: https://docs.traefik.io/v1.7/basics/#collected-data\n"
time="2019-11-25T08:25:30Z" level=info msg="Preparing server https &{Address::443 TLS:0xc0002b22d0 Redirect: Auth: WhitelistSourceRange:[] WhiteList: Compress:false ProxyProtocol: ForwardedHeaders:0xc00050a100} with readTimeout=0s writeTimeout=0s idleTimeout=3m0s"
time="2019-11-25T08:25:30Z" level=info msg="Preparing server traefik &{Address::8080 TLS: Redirect: Auth: WhitelistSourceRange:[] WhiteList: Compress:false ProxyProtocol: ForwardedHeaders:0xc00050a120} with readTimeout=0s writeTimeout=0s idleTimeout=3m0s"
time="2019-11-25T08:25:30Z" level=info msg="Starting server on :443"
time="2019-11-25T08:25:30Z" level=info msg="Preparing server http &{Address::80 TLS: Redirect:0xc000843fc0 Auth: WhitelistSourceRange:[] WhiteList: Compress:false ProxyProtocol: ForwardedHeaders:0xc00050a0e0} with readTimeout=0s writeTimeout=0s idleTimeout=3m0s"
time="2019-11-25T08:25:30Z" level=info msg="Starting server on :8080"
time="2019-11-25T08:25:30Z" level=info msg="Starting provider configuration.ProviderAggregator {}"
time="2019-11-25T08:25:30Z" level=info msg="Starting server on :80"
time="2019-11-25T08:25:30Z" level=info msg="Starting provider *kubernetes.Provider {\"Watch\":true,\"Filename\":\"\",\"Constraints\":[],\"Trace\":false,\"TemplateVersion\":0,\"DebugLogGeneratedTemplate\":false,\"Endpoint\":\"\",\"Token\":\"\",\"CertAuthFilePath\":\"\",\"DisablePassHostHeaders\":false,\"EnablePassTLSCert\":false,\"Namespaces\":null,\"LabelSelector\":\"\",\"IngressClass\":\"\",\"IngressEndpoint\":null,\"ThrottleDuration\":0}"
time="2019-11-25T08:25:30Z" level=info msg="ingress label selector is: \"\""
time="2019-11-25T08:25:30Z" level=info msg="Creating in-cluster Provider client"
time="2019-11-25T08:25:30Z" level=info msg="Server configuration reloaded on :443"
time="2019-11-25T08:25:30Z" level=info msg="Server configuration reloaded on :8080"
time="2019-11-25T08:25:30Z" level=info msg="Server configuration reloaded on :80"
3.4、Rewrite
测试demo：
apiVersion: v1
kind: Service
metadata:
name: nginx-demo
spec:
selector:
app: nginx
ports:
- name: nginx
port: 80
---
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: nginx
labels:
app: nginx
spec:
replicas: 2
selector:
matchLabels:
app: nginx
template:
metadata:
labels:
app: nginx
spec:
containers:
- name: nginx
image: nginx
imagePullPolicy: IfNotPresent
正常访问的ingress如下：
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: nginx-ingress-demo
annotations:
kubernetes.io/ingress.class: "traefik"
spec:
rules:
- host:
http:
paths:
- backend:
serviceName: nginx-demo
servicePort: 80
path: /
但是如果我们把path改为如下：
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: nginx-ingress-demo
annotations:
kubernetes.io/ingress.class: "traefik"
spec:
rules:
- host:
http:
paths:
- backend:
serviceName: nginx-demo
servicePort: 80
path: /app
那么这时候访问就会报404.
image.png
这时候我们可以在配置里加上rewrite规则，如下：
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: nginx-ingress-demo
annotations:
kubernetes.io/ingress.class: "traefik"
traefik.ingress.kubernetes.io/rewrite-target: /
spec:
rules:
- host:
http:
paths:
- backend:
serviceName: nginx-demo
servicePort: 80
path: /app
然后就可以正常访问了，如下：
image.png
为了更友好化，我们可以定义如下：
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: nginx-ingress-demo
annotations:
kubernetes.io/ingress.class: "traefik"
traefik.ingress.kubernetes.io/rewrite-target: /$2
spec:
rules:
- host:
http:
paths:
- backend:
serviceName: nginx-demo
servicePort: 80
path: /app(/|$)(.*)
但是这种还是会有问题，比如有的静态资源无法正常显示，比如静态资源的路径在/static下面，现在我们做了 url rewrite 过后，要正常访问也需要带上前缀才可以：http://xxx.xxx/static/screen.css，对于图片或者其他静态资源也是如此，当然我们去更改页面引入静态资源的方式为相对路径也是可以的，但是毕竟要修改代码，这个时候我们可以借助 ingress-traefik 中的 configuration-snippet 来对静态资源做一次跳转，如下所示：
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: nginx-ingress-demo
annotations:
kubernetes.io/ingress.class: "traefik"
traefik.ingress.kubernetes.io/rewrite-target: /$2
traefik.ingress.kubernetes.io/configuration-snippet: |
rewrite ^/static/(.*)$ /app/static/$1 redirect;
rewrite ^/image/(.*)$ /app/image/$1 redirect;
spec:
rules:
- host:
http:
paths:
- backend:
serviceName: nginx-demo
servicePort: 80
path: /app(/|$)(.*)
这时候就可以正常加载静态资源了。
但是我们还想直接访问主域名，但是会报404，要解决这种问题，可以app-root的注解，这样我们访问主域名的时候就可以跳转到app-root目录（真实存在的路径）下面，如下：
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: nginx-ingress-demo
annotations:
kubernetes.io/ingress.class: "traefik"
traefik.ingress.kubernetes.io/app-root: /app/
traefik.ingress.kubernetes.io/rewrite-target: /$2
traefik.ingress.kubernetes.io/configuration-snippet: |
rewrite ^/static/(.*)$ /app/static/$1 redirect;
rewrite ^/image/(.*)$ /app/image/$1 redirect;
spec:
rules:
- host:
http:
paths:
- backend:
serviceName: nginx-demo
servicePort: 80
path: /app(/|$)(.*)
但是还有一个问题是我们的 path 路径其实也匹配了 /app 这样的路径，可能我们更加希望我们的应用在最后添加一个 / 这样的 slash，同样我们可以通过 configuration-snippet 配置来完成，如下 Ingress 对象：
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: nginx-ingress-demo
annotations:
kubernetes.io/ingress.class: "traefik"
traefik.ingress.kubernetes.io/app-root: /app/
traefik.ingress.kubernetes.io/rewrite-target: /$2
traefik.ingress.kubernetes.io/configuration-snippet: |
rewrite ^(/app)$ $1/ redirect;
rewrite ^/static/(.*)$ /app/static/$1 redirect;
rewrite ^/image/(.*)$ /app/image/$1 redirect;
spec:
rules:
- host:
http:
paths:
- backend:
serviceName: nginx-demo
servicePort: 80
path: /app(/|$)(.*)
-----------------------
公众号：乔边故事（ID：qiaobiangushi）
知乎: 乔边故事
头条号：乔边故事
只要脸皮够厚，整个世界都将被你踩在脚下。
-----------------------