k8s笔记（6）-- Ingress和Ingress Controller
什么是Ingress
Ingress是k8s的一个资源对象，作为一个集群服务的入口，用于管理对集群中服务的外部访问。
可以将Ingress配置为提供服务外部访问的URL，负载均衡，绑定SSL / TLS，并提供基于名称的虚拟主机。
作为集群服务的入口，Ingress类似一个代理，处于多个service的前端，不会公开任意端口或协议，通常会作为HTTP（s）负载均衡器转发http(s)流量。而要将HTTP和HTTPS以外的服务公开到Internet时，通常使用NodePort或 LoadBalancer 类型的服务。
Ingress Nginx
为了使Ingress正常工作，集群必须运行一个Ingress Controller。Ingress Controller是一个部署为Kubernetes Pod的守护程序，它监视apiserver的ingresses endpoint 以获取对Ingress 资源的更新。k8s支持多个ingress controller，这里仅介绍使用ingress nginx。
ingress-nginx是围绕Ingress构建的，使用ConfigMap存储NGINX配置，会动态生效ingress中定义的规则。
ingress-nginx部署：
# 安装
kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/static/mandatory.yaml
# 查看
kubectl get pods -n ingress-nginx -l app.kubernetes.io/name=ingress-nginx -w
# 查看ingress-nginx版本
POD_NAMESPACE=ingress-nginx
POD_NAME=$(kubectl get pods -n $POD_NAMESPACE -l app.kubernetes.io/name=ingress-nginx -o jsonpath='{.items[0].metadata.name}')
kubectl exec -it $POD_NAME -n ingress-nginx -- /nginx-ingress-controller --version
# 创建ingress-ingress的service，service给ingress提供ip和代理ingress端口映射，这里直接使用的是官方的yaml
kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/static/provider/baremetal/service-nodeport.yaml
# kubectl get svc -n ingress-nginx
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
ingress-nginx NodePort 10.10.150.34 80:32227/TCP,443:30117/TCP 15s
Ingress 实践
先部署后端service：hostnamev1，hostnamev2，app
# cat hostnamev1-demo.yaml
apiVersion: v1
kind: Service
metadata:
name: hostnamev1
namespace: default
spec:
selector:
demo: hostnamev1
release: canary
ports:
- name: http
port: 80
targetPort: 8000
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: hostnamev1-deploy
namespace: default
spec:
replicas: 2
selector:
matchLabels:
demo: hostnamev1
release: canary
template:
metadata:
labels:
demo: hostnamev1
release: canary
spec:
containers:
- name: hostnamev1
image: xlbubble/hostname:1.0
ports:
- name: http
containerPort: 8000
# hostnamev2-demo.yaml
apiVersion: v1
kind: Service
metadata:
name: hostnamev2
namespace: default
spec:
selector:
demo: hostnamev2
release: canary
ports:
- name: http
port: 80
targetPort: 8000
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: hostnamev2-deploy
namespace: default
spec:
replicas: 2
selector:
matchLabels:
demo: hostnamev2
release: canary
template:
metadata:
labels:
demo: hostnamev2
release: canary
spec:
containers:
- name: hostnamev2
image: xlbubble/hostname:2.0
ports:
- name: http
containerPort: 8000
# cat app-demo.yaml
apiVersion: v1
kind: Service
metadata:
name: app
namespace: default
spec:
selector:
demo: app
ports:
- name: http
port: 8080
targetPort: 8080
---
apiVersion: v1
kind: Pod
metadata:
name: app
namespace: default
labels:
demo: app
spec:
containers:
- name: app
image: tomcat:9.0.30-jdk8-openjdk-slim
ports:
- name: http
containerPort: 8080
# kubectl apply -f hostnamev1-demo.yaml
# kubectl apply -f hostnamev2-demo.yaml
# kubectl apply -f app-demo.yaml
部署http-ingress做代理
# kubectl explain ingress.spec.rules
# cat http-ingress.yaml
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: http-ingress
namespace: default
annotations:
kubernetes.io/ingress.class: 'nginx'
spec:
rules: # 定义后端转发规则
- host: www.xlbubble.xyz # 通过www域名进行转发
http:
paths:
- path: # 配置访问路径
backend: # 配置后端服务
serviceName: hostnamev1 # 绑定后端service hostnamev1
servicePort: 80 # 指定service端口
- path: /app
backend:
serviceName: app
servicePort: 8080
- host: test.xlbubble.xyz
http:
paths:
- path:
backend:
serviceName: hostnamev2
servicePort: 80
# kubectl apply -f http-ingress.yaml
describe ingress看下对应关系
# kubectl get ingress
NAME HOSTS ADDRESS PORTS AGE
http-ingress www.xlbubble.xyz,test.xlbubble.xyz 10.10.150.34 80 14m
# kubectl get pods -o wide
NAME READY STATUS RESTARTS AGE IP
app 1/1 Running 0 34m 10.244.2.56
hostnamev1-deploy-cd74d7fbd-82k4q 1/1 Running 0 106m 10.244.2.53
hostnamev1-deploy-cd74d7fbd-lmr6m 1/1 Running 0 106m 10.244.1.55
hostnamev2-deploy-759d846c95-k7ckn 1/1 Running 0 61m 10.244.1.56
hostnamev2-deploy-759d846c95-q9xf9 1/1 Running 0 61m 10.244.2.54
# kubectl describe ingress http-ingress
Name: http-ingress
Namespace: default
Address: 10.10.150.34
Default backend: default-http-backend:80 ()
Rules: # ingress配置规则
Host Path Backends
---- ---- --------
www.xlbubble.xyz
hostnamev1:80 (10.244.1.55:8000,10.244.2.53:8000)
/app app:8080 (10.244.2.56:8080)
test.xlbubble.xyz
hostnamev2:80 (10.244.1.56:8000,10.244.2.54:8000)
...
访问测试
# curl www.xlbubble.xyz
Version: 1.0 | hostnamev1-deploy-cd74d7fbd-82k4q
# curl test.xlbubble.xyz
Version: 2.0 | hostnamev2-deploy-759d846c95-k7ckn
# 访问app 404,这里访问www.xlbubble.xyz/app会默认转发到后端10.244.2.56:8080/app上，也就是带着路径转发，如果要直接转发到10.244.2.56:8080，需要额外配置，见下文ingress-nginx rewrite转发
# curl www.xlbubble.xyz/app -I
HTTP/1.1 404
Server: openresty/1.15.8.2
Date: Fri, 03 Dec 2019 04:10:38 GMT
Content-Type: text/html;charset=utf-8
Connection: keep-alive
Vary: Accept-Encoding
Content-Language: en
ingress-nginx rewrite转发
# kubectl edit ingress http-ingress
#
...
annotations:
kubernetes.io/ingress.class: 'nginx'
nginx.ingress.kubernetes.io/rewrite-target: /$2
spec:
rules:
- host: www.xlbubble.xyz
http:
paths:
- path:
backend:
serviceName: hostnamev1
servicePort: 80
- path: /app(/|$)(.*)
backend:
serviceName: app
servicePort: 8080
# 访问测试,访问www.xlbubble.xyz/app转发到了10.244.2.56:8080/
# curl www.xlbubble.xyz/app -I
HTTP/1.1 200
Server: openresty/1.15.8.2
Date: Fri, 03 Dec 2019 09:43:49 GMT
Content-Type: text/html;charset=UTF-8
Connection: keep-alive
Vary: Accept-Encoding
参数解释:
大致意思就是将/app(/|$)(.*)重写为/$2,如/app--> /，/app/zoneinfo--> /zoneinfo。
nginx.ingress.kubernetes.io/rewrite-target: /$2：必须将流量重定向到的目标URI（/$2），$2表示第二个参数，就下面这第二个括号里的
path: /app(/|$)(.*)：(.*)中的所有参数都将传给$2
ingress-nginx控制器中的nginx配置文件，可以看到有写入对应的nginx配置
# kubectl exec -n ingress-nginx -it nginx-ingress-controller-568867bf56-ghvvv bash
$ cat nginx.conf
...
## start server_ # ingress-nginx默认配置
...
## end server_
## start server test.xlbubble.xyz
server {
server_name test.xlbubble.xyz ;
listen 80 ;
listen 443 ssl http2 ; # 已经配置了ssl，就差http-ingress.yaml中配置和挂载ssl证书了
set $proxy_upstream_name "-";
ssl_certificate_by_lua_block {
certificate.call()
}
location / { # http-ingress中定义的配置
set $namespace "default";
set $ingress_name "http-ingress";
set $service_name "hostnamev2";
set $service_port "80";
set $location_path "/";
...
}
}
## end server test.xlbubble.xyz
## start server www.xlbubble.xyz
server {
server_name www.xlbubble.xyz ;
listen 80 ;
listen 443 ssl http2 ;
set $proxy_upstream_name "-";
ssl_certificate_by_lua_block {
certificate.call()
}
location /app {
set $namespace "default";
set $ingress_name "http-ingress";
set $service_name "app";
set $service_port "8080";
set $location_path "/app";
...
}
location / {
set $namespace "default";
set $ingress_name "http-ingress";
set $service_name "hostnamev1";
set $service_port "80";
set $location_path "/";
...
}
}
## end server www.xlbubble.xyz
给hostnamev1配置https
# 创建secret，我这里用的是腾讯云的域名+证书，没有证书可以自己生成。
kubectl create secret tls tomcat-ingress-secret --cert=1_www.xlbubble.xyz_bundle.crt --key=2_www.xlbubble.xyz.key
# cat http-ingress-tls.yaml
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: http-ingress
namespace: default
annotations:
kubernetes.io/ingress.class: 'nginx'
spec:
tls: # 配置ssl证书
- hosts:
- www.xlbubble.xyz # 指定证书域名
secretName: http-ingress-secret # 绑定证书
rules:
- host: www.xlbubble.xyz
http:
paths:
- path:
backend:
serviceName: hostnamev1
servicePort: 80
访问测试
# curl https://www.xlbubble.xyz -I
HTTP/1.1 200 OK
Server: openresty/1.15.8.2
Date: Fri, 03 Dec 2019 11:01:03 GMT
Content-Type: text/html; charset=utf-8
Content-Length: 59
Connection: keep-alive
Strict-Transport-Security: max-age=15724800; includeSubDomains
X-Powered-By: Express
ETag: W/"3b-mLuijDfag/PvLxTOq9nzeLxp0gQ"
# http访问发现，ingress-nginx自动配置了http跳转https
# curl www.xlbubble.xyz -I
HTTP/1.1 308 Permanent Redirect
Server: openresty/1.15.8.2
Date: Fri, 03 Dec 2019 11:16:32 GMT
Content-Type: text/html
Content-Length: 177
Connection: keep-alive
Location: https://www.xlbubble.xyz/
参考和扩展
k8s官方-ingress