K8s ingress资源和ingress controller
什么是Ingress？
Ingress：就是能利用 Nginx（不常用）、Haproxy（不常用）、Traefik（常用）、Envoy（常用） 等负载均衡器暴露集群内服务的工具。Ingress提供七层负载均衡能力，可以通过 Ingress 配置提供外部可访问的 URL、负载均衡、SSL、基于名称的虚拟主机等。作为集群流量接入层，Ingress 的高可靠性显得尤为重要。
我们知道service的表现形式为IP:PORT，即工作在第四层传输层（TCP/IP层），那么对于不同的URL地址经常对应用不同的后端服务或者虚拟服务器，这些应用层的转发机制仅通过kubernetes的service机制是无法实现的，这种情况我么可以使用ingress策略定义和一个具体的ingress Controller，两者结合实现一个完整的Ingress 负载均衡，这个负载均衡是基于nginx七层反向代理来实现的，ingress工作原理如下图：
外部客户端通过访问负载均衡器，然后调度到service上，然后在调度到IngressController，IngressController通过Ingress规则（域名或虚拟主机）访问到后端pod，而在Ingress规则当中对应的主机是又service分组来设定的，可以看到，这幅图有2种service，最上面的service是用来对外提供服务的，而下面2个service是仅仅对后端pod分组,不被调度时使用,如果后端pod发生变动,则ingress就会将变动信息注入到,ingress controller管理的7层负载nginx的配置文件中.。
Ingress-nginx项目地址 ：https://github.com/kubernetes/ingress-nginx
部署Ingress控制器
下载mandatory.yaml文件
[root@s1 ~]# wget https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/static/mandatory.yaml部署
[root@s1 ~]# kubectl apply -f mandatory.yaml
namespace/ingress-nginx created
configmap/nginx-configuration created
configmap/tcp-services created
configmap/udp-services created
serviceaccount/nginx-ingress-serviceaccount created
clusterrole.rbac.authorization.k8s.io/nginx-ingress-clusterrole created
role.rbac.authorization.k8s.io/nginx-ingress-role created
rolebinding.rbac.authorization.k8s.io/nginx-ingress-role-nisa-binding created
clusterrolebinding.rbac.authorization.k8s.io/nginx-ingress-clusterrole-nisa-binding created
deployment.apps/nginx-ingress-controller created[root@s1 ~]# kubectl get pods -n ingress-nginx -w -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
nginx-ingress-controller-568867bf56-6gxdx 0/1 ContainerCreating 0 59s n1
nginx-ingress-controller-568867bf56-6gxdx 0/1 Running 0 61s 10.244.1.28 n1
nginx-ingress-controller-568867bf56-6gxdx 1/1 Running 0 65s 10.244.1.28 n1 nginx-ingress-controller部署在n1上,一个deployment控制器,一个replicaset,一个pod。
接下来还需要部署一个service-nodeport服务,才能实现把集群外部流量接入到集群中来:
[root@s1 ~]# wget https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/baremetal/service-nodeport.yaml在文件中指定一下nodeport方便部署应用，修改文件中加两个nodePort参数，并为service设置已部署ingress的pod标签，最终如下：
[root@s1 ~]# vim service-nodeport.yaml
apiVersion: v1
kind: Service
metadata:
name: ingress-nginx
namespace: ingress-nginx
labels:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
spec:
type: NodePort
ports:
- name: http
port: 80
targetPort: 80
protocol: TCP
nodePort: 30080
- name: https
port: 443
targetPort: 443
protocol: TCP
nodePort: 30443
selector:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx[root@s1 ~]# kubectl apply -f service-nodeport.yaml
service/ingress-nginx created
[root@s1 ~]# kubectl get svc -n ingress-nginx
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
ingress-nginx NodePort 10.96.157.3 80:30080/TCP,443:30443/TCP 50s 我们直接通过n1节点的ip就可以访问到应用：
[root@s1 ~]# curl 192.168.100.50:30080
404 Not Found
404 Not Found
openresty/1.15.8.2
因为后端没有实际应用的80端口pod，结果自然就返回404。
接下来就部署后端service及pod：
示例一：
定义myapp service
[root@s1 ingress]# vim deploy-demo.yaml
apiVersion: v1
kind: Service
metadata:
name: myapp-svc
namespace: default
spec:
selector:
app: myapp
release: canary
clusterIP: "None"
ports:
- port: 80
targetPort: 80
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: myapp-deploy
namespace: default
spec:
replicas: 2
selector:
matchLabels:
app: myapp
release: canary
template:
metadata:
labels:
app: myapp
release: canary
spec:
containers:
- name: myapp
image: ikubernetes/myapp:v1
ports:
- name: http
containerPort: 80[root@s1 ingress]# kubectl apply -f deploy-demo.yaml
service/myapp-svc created
deployment.apps/myapp-deploy created
[root@s1 ingress]# kubectl get svc
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
kubernetes ClusterIP 10.96.0.1 443/TCP 12d
myapp-svc ClusterIP None 80/TCP 12s
[root@s1 ingress]# kubectl get pods
NAME READY STATUS RESTARTS AGE
myapp-deploy-7d574d56c7-9p5ql 1/1 Running 0 30s
myapp-deploy-7d574d56c7-j6xkk 1/1 Running 0 30s 把myapp service通过ingress发布出去
下面我们再定义一个清单文件，把myapp应用通过Ingress（相当于nginx的反向代理功能）发布出去：
[root@s1 ingress]# vim ingress-myapp.yaml
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: ingress-myapp
namespace: default #要和deployment和要发布的service处于同一个名称空间
annotations: #这个注解说明我们要用到的ingress-controller是nginx，而不是traefic，enjoy
kubernetes.io/ingress.class: "nginx"
spec:
rules:
- host: myapp.lemon.com #表示访问这个域名，就会转发到后端myapp管理的pod上的服务
http:
paths:
- path:
backend:
serviceName: myapp-svc
servicePort: 80[root@s1 ingress]# kubectl apply -f ingress-myapp.yaml
ingress.extensions/ingress-myapp created
[root@s1 ingress]# kubectl describe ingress
Name: ingress-myapp
Namespace: default
Address:
Default backend: default-http-backend:80 ()
Rules:
Host Path Backends
---- ---- --------
myapp.lemon.com
myapp-svc:80 (10.244.1.29:80,10.244.2.26:80)
Annotations:
kubernetes.io/ingress.class: nginx
kubectl.kubernetes.io/last-applied-configuration: {"apiVersion":"extensions/v1beta1","kind":"Ingress","metadata":{"annotations":{"kubernetes.io/ingress.class":"nginx"},"name":"ingress-myapp","namespace":"default"},"spec":{"rules":[{"host":"myapp.lemon.com","http":{"paths":[{"backend":{"serviceName":"myapp-svc","servicePort":80},"path":null}]}}]}}
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal CREATE 2m32s nginx-ingress-controller Ingress default/ingress-myapp
[root@s1 ingress]# kubectl get pods -n ingress-nginx
NAME READY STATUS RESTARTS AGE
nginx-ingress-controller-568867bf56-6gxdx 1/1 Running 0 进入ingress-controller交互式命令行里面，可以清晰的看到nginx是怎么反向代理我们myapp.lemon.com的：
[root@s1 ~]# kubectl exec -n ingress-nginx -it nginx-ingress-controller-568867bf56-6gxdx -- /bin/sh
$cat nginx.conf
外部浏览器中访问，正确返回pod数据。如图：
示例二：
将tomcat服务添加至ingress-nginx中
[root@s1 ingress]# vim tomcat-demo.yaml
apiVersion: v1
kind: Service
apiVersion: v1
kind: Service
metadata:
name: tomcat
namespace: default
spec:
selector:
app: tomcat
ports:
- name: http
port: 8080
targetPort: 8080
- name: ajp
port: 8009
targetPort: 8009
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: tomcat
namespace: default
spec:
replicas: 2
selector:
matchLabels:
app: tomcat
template:
metadata:
labels:
app: tomcat
spec:
containers:
- name: tomcat
image: tomcat:7-alpine
imagePullPolicy: IfNotPresent
ports:
- name: http
containerPort: 8080
- name: ajp
containerPort: 8009[root@s1 ingress]# kubectl apply -f tomcat-demo.yaml
service/tomcat created
deployment.apps/tomcat created
[root@s1 ingress]# kubectl get svc
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
kubernetes ClusterIP 10.96.0.1 443/TCP 12d
myapp-svc ClusterIP None 80/TCP 46m
tomcat ClusterIP 10.99.236.187 8080/TCP,8009/TCP 10s
[root@s1 ingress]# kubectl get pods
NAME READY STATUS RESTARTS AGE
myapp-deploy-7d574d56c7-9p5ql 1/1 Running 0 49m
myapp-deploy-7d574d56c7-j6xkk 1/1 Running 0 49m
tomcat-59f87c8677-66lkn 1/1 Running 0 3m22s
tomcat-59f87c8677-zvpr9 1/1 Running 0 3m22s 将tomcat服务添加至ingress-nginx中
[root@s1 ingress]# vim ingress-tomcat.yaml
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: ingress-mytomcat
namespace: default
annotations:
kubernetes.io/ingress.class: "nginx"
spec:
rules:
- host: mytomcat.lemon.com
http:
paths:
- path:
backend:
serviceName: tomcat
servicePort: 8080[root@s1 ingress]# kubectl apply -f ingress-tomcat.yaml
ingress.extensions/ingress-mytomcat created
[root@s1 ingress]# kubectl get ingress
NAME HOSTS ADDRESS PORTS AGE
ingress-myapp myapp.lemon.com 10.96.157.3 80 37m
ingress-mytomcat mytomcat.lemon.com 10.96.157.3 80 35s可以看到ingress控制器已经创建。
[root@s1 ingress]# kubectl describe ingress
Name: ingress-myapp
Namespace: default
Address: 10.96.157.3
Default backend: default-http-backend:80 ()
Rules:
Host Path Backends
---- ---- --------
myapp.lemon.com
myapp-svc:80 (10.244.1.29:80,10.244.2.26:80)
Annotations:
kubectl.kubernetes.io/last-applied-configuration: {"apiVersion":"extensions/v1beta1","kind":"Ingress","metadata":{"annotations":{"kubernetes.io/ingress.class":"nginx"},"name":"ingress-myapp","namespace":"default"},"spec":{"rules":[{"host":"myapp.lemon.com","http":{"paths":[{"backend":{"serviceName":"myapp-svc","servicePort":80},"path":null}]}}]}}
kubernetes.io/ingress.class: nginx
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal CREATE 42m nginx-ingress-controller Ingress default/ingress-myapp
Normal UPDATE 41m nginx-ingress-controller Ingress default/ingress-myapp
Name: ingress-mytomcat
Namespace: default
Address: 10.96.157.3
Default backend: default-http-backend:80 ()
Rules:
Host Path Backends
---- ---- --------
mytomcat.lemon.com
tomcat:8080 (10.244.1.30:8080,10.244.2.27:8080)
Annotations:
kubectl.kubernetes.io/last-applied-configuration: {"apiVersion":"extensions/v1beta1","kind":"Ingress","metadata":{"annotations":{"kubernetes.io/ingress.class":"nginx"},"name":"ingress-mytomcat","namespace":"default"},"spec":{"rules":[{"host":"mytomcat.lemon.com","http":{"paths":[{"backend":{"serviceName":"tomcat","servicePort":8080},"path":null}]}}]}}
kubernetes.io/ingress.class: nginx
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal CREATE 6m13s nginx-ingress-controller Ingress default/ingress-mytomcat
Normal UPDATE 5m44s nginx-ingress-controller Ingress default/ingress-mytomcat 访问Tomcat资源看看：
基于ssl协议的访问
先做个自签的证书
[root@s1 ingress]# openssl genrsa -out tls.key 2048
Generating RSA private key, 2048 bit long modulus
.......................................+++
.......................................+++
e is 65537 (0x10001)
[root@s1 ingress]# openssl req -new -x509 -key tls.key -out tls.crt -subj /C=CN/ST=Beijing/O=DevOps/CN=mytomcat.lemon.com
[root@s1 ingress]# ls
deploy-demo.yaml ingress-myapp.yaml ingress-tomcat.yaml tls.crt tls.key tomcat-demo.yaml创建secret
[root@s1 ingress]# kubectl create secret tls mytomcat-ingress-secret --cert=tls.crt --key=tls.key
secret/mytomcat-ingress-secret created查看证书
[root@s1 ingress]# kubectl describe secret mytomcat-ingress-secret
Name: mytomcat-ingress-secret
Namespace: default
Labels:
Annotations:
Type: kubernetes.io/tls
Data
====
tls.crt: 1249 bytes
tls.key: 1675 bytes 通过secret把证书注入到pod中。
[root@s1 ingress]# vim ingress-tomcat-tls.yaml
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: ingress-mytomcat-tls
namespace: default
annotations:
kubernetes.io/ingress.class: "nginx"
spec:
tls:
- hosts:
- mytomcat.lemon.com #这里写域名
secretName: mytomcat-ingress-secret #这里写secret证书名称
rules:
- host: mytomcat.lemon.com
http:
paths:
- path:
backend:
serviceName: tomcat
servicePort: 8080
[root@s1 ingress]# kubectl apply -f ingress-tomcat-tls.yaml
ingress.extensions/ingress-mytomcat-tls created 访问https资源，pod上定义的30443端口：
