Kubernetes 初识Ingress Controller以及部署
在Kubernetes集群中，Ingress对集群服务（Service）中外部可访问的API对象进行管理，提供七层负载均衡能力。
Ingress基本概念
在Kubernetes集群中，Ingress作为集群内服务对外暴露的访问接入点，其几乎承载着集群内服务访问的所有流量。Ingress是Kubernetes中的一个资源对象，用来管理集群外部访问集群内部服务的方式。您可以通过Ingress资源来配置不同的转发规则，从而达到根据不同的规则设置访问集群内不同的Service所对应的后端Pod。
Ingress资源仅支持配置HTTP流量的规则，无法配置一些高级特性，例如负载均衡的算法、Sessions Affinity等，这些高级特性都需要在Ingress Controller中进行配置。
Nginx Ingress Controller
当前Kubernetes官方维护的是Nginx Ingress Controller，ACK基于社区版的Nginx Ingress Controller进行了优化。ACK的Nginx Ingress Controller与社区Nginx Ingress Controller完全兼容，支持社区所有的Annonation。当在创建ACK集群时，您选择安装的Nginx Ingress Controller组件即为ACK定制版的Nginx Ingress Controller组件。
工作原理
为了使得Nginx Ingress资源正常工作，集群中必须要有个Nginx Ingress Controller来解析Nginx Ingress的转发规则。Nginx Ingress Controller收到请求，匹配Nginx Ingress转发规则转发到后端Service所对应的Pod，由Pod处理请求。Kubernetes中Service、Nginx Ingress与Nginx Ingress Controller有着以下关系：
- Service是后端真实服务的抽象，一个Service可以代表多个相同的后端服务。
- Nginx Ingress是反向代理规则，用来规定HTTP/HTTPS请求应该被转发到哪个Service所对应的Pod上。例如根据请求中不同的Host和URL路径，让请求落到不同Service所对应的Pod上。
- Nginx Ingress Controller是一个反向代理程序，负责解析Nginx Ingress的反向代理规则。如果Nginx Ingress有增删改的变动，Nginx Ingress Controller会及时更新自己相应的转发规则，当Nginx Ingress Controller收到请求后就会根据这些规则将请求转发到对应Service的Pod上。
Nginx Ingress Controller通过API Server获取Ingress资源的变化，动态地生成Load Balancer（例如Nginx）所需的配置文件（例如nginx.conf），然后重新加载Load Balancer（例如执行nginx -s load重新加载Nginx）来生成新的路由转发规则。
Nginx Ingress Controller可通过配置LoadBalancer类型的Service来创建SLB，因此可以从外部通过SLB访问到Kubernetes集群的内部服务。根据Nginx Ingress配置的不同规则来访问不同的服务。
Ingress是什么
NodePort存在的不足：
• 一个端口只能一个服务使用，端口需提前规划
• 只支持4层负载均衡（iptables ipvs）
当应用越来越多的时候，管理端口就会变得麻烦起来了。那么可不可以在集群当中暴露一个端口完成所有项目的统一接入。
Nodeport只支持四层的数据包转发，所以不支持基于域名的做分流，基于uri做匹配。
有没有只在集群当中暴露一个端口同时是一个全局的负载均衡器可以提供所有项目的统一接入呢？
Ingress：Ingress公开了从集群外部到集群内服务的HTTP和HTTPS路由的规则集合，而具体实现流量路由则是由Ingress Controller负责。
Ingress
• Ingress：K8s中的一个抽象资源，给管理员提供一个暴露应用的入口定义方法
• Ingress Controller：根据Ingress生成具体的路由规则，并对Pod负载均衡器
Ingress Controller
Ingress Controller有很多实现，我们这里采用官方维护的Nginx控制器。
- 项目地址：https://github.com/kubernetes/ingress-nginx
- 部署：kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/nginx-0.30.0/deploy/static/mandatory.yaml
注意事项：
- 将Ingress Controller暴露，一般使用宿主机网络（hostNetwork: true）或者使用NodePort
其他控制器： https://kubernetes.io/docs/concepts/services-networking/ingress-controllers/
[root@k8s-master ~]# wget https://raw.githubusercontent.com/kubernetes/ingress-nginx/nginx-0.30.0/deploy/static/mandatory.yaml这里镜像quay.io/kubernetes-ingress-controller/nginx-ingress-controller:0.30.0，可以使用docker pull手工去拉取如果拉取不下来使用这个地址lizhenliang/nginx-ingress-controller:0.30.0
在部署Ingress Controller yml需要注意的地方
（1）启动程序的几个参数，其中的变量使用env定义
args:
- /nginx-ingress-controller
- --configmap=$(POD_NAMESPACE)/nginx-configuration
- --tcp-services-configmap=$(POD_NAMESPACE)/tcp-services
- --udp-services-configmap=$(POD_NAMESPACE)/udp-services
- --publish-service=$(POD_NAMESPACE)/ingress-nginx
- --annotations-prefix=nginx.ingress.kubernetes.io
env:
- name: POD_NAME
valueFrom:
fieldRef:
fieldPath: metadata.name
- name: POD_NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
（2）可不可以不创建service使用其他方法使得其可以暴露在集群之外
[root@k8s-master ~]# kubectl get svc -n ingress-nginx
No resources found in ingress-nginx namespace.在container加上这个字段(这里可以看到，这里并没有为我们生成service，所以先要使用nodeport暴露出去,这里不使用nodeport)
spec:
hostNetwork: true不会独立为其分配网络，也就是不会创建infocontainer这个容器了。这个pod在哪个节点上就使用哪个节点的网络
[root@k8s-master ~]# kubectl get pod -n ingress-nginx -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
nginx-ingress-controller-q6gp7 1/1 Running 0 122m 192.168.179.104 k8s-node2
nginx-ingress-controller-xq7fl 1/1 Running 0 122m 192.168.179.103 k8s-node1 （3）提供的端口为80 443来统一接入流量
ports:
- name: http
containerPort: 80
protocol: TCP
- name: https
containerPort: 443
protocol: TCP[root@k8s-node1 ~]# netstat -tpln | grep -wE "80|443"
tcp 0 0 0.0.0.0:443 0.0.0.0:* LISTEN 32669/nginx: master
tcp 0 0 0.0.0.0:80 0.0.0.0:* LISTEN 32669/nginx: master
tcp6 0 0 :::443 :::* LISTEN 32669/nginx: master
tcp6 0 0 :::80 :::* LISTEN 32669/nginx: master
[root@k8s-node2 ~]# netstat -tpln | grep -wE "80|443"
tcp 0 0 0.0.0.0:80 0.0.0.0:* LISTEN 27102/nginx: master
tcp 0 0 0.0.0.0:443 0.0.0.0:* LISTEN 27102/nginx: master
tcp6 0 0 :::80 :::* LISTEN 27102/nginx: master
tcp6 0 0 :::443 :::* LISTEN 27102/nginx: master (4)健康检查，这里健康检查提供了一个uri路径/healthz
livenessProbe:
failureThreshold: 3
httpGet:
path: /healthz
port: 10254
scheme: HTTP
initialDelaySeconds: 10
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 10
readinessProbe:
failureThreshold: 3
httpGet:
path: /healthz
port: 10254
scheme: HTTP
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 10[root@k8s-master ~]# curl -I 192.168.179.103:80/healthz
HTTP/1.1 200 OK
Server: nginx/1.17.8
Date: Tue, 29 Dec 2020 16:41:56 GMT
Content-Type: text/html
Content-Length: 0
Connection: keep-alive(5)使用钩子在程序关闭之前做了一定的处理
lifecycle:
preStop:
exec:
command:
- /wait-shutdown(6)对资源的限制
apiVersion: v1
kind: LimitRange
metadata:
name: ingress-nginx
namespace: ingress-nginx
labels:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
spec:
limits:
- min:
memory: 90Mi
cpu: 100m
type: Container(7)我这里修改了yml使用了demoset
apiVersion: apps/v1
kind: DaemonSet
metadata:
name: nginx-ingress-controller
namespace: ingress-nginx
上面Ingress Controller搭建好，因为这里还没有具体创建规则，在使用ingress controll之前要把deployment控制器部署在集群当中，部署好之后创建ingress规则。
上面就算部署好了Ingress Controller，具体yml文件内容如下:
[root@k8s-master ~]# cat ingress-controller.yaml
apiVersion: v1
kind: Namespace
metadata:
name: ingress-nginx
labels:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
---
kind: ConfigMap
apiVersion: v1
metadata:
name: nginx-configuration
namespace: ingress-nginx
labels:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
---
kind: ConfigMap
apiVersion: v1
metadata:
name: tcp-services
namespace: ingress-nginx
labels:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
---
kind: ConfigMap
apiVersion: v1
metadata:
name: udp-services
namespace: ingress-nginx
labels:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: nginx-ingress-serviceaccount
namespace: ingress-nginx
labels:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRole
metadata:
name: nginx-ingress-clusterrole
labels:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
rules:
- apiGroups:
- ""
resources:
- configmaps
- endpoints
- nodes
- pods
- secrets
verbs:
- list
- watch
- apiGroups:
- ""
resources:
- nodes
verbs:
- get
- apiGroups:
- ""
resources:
- services
verbs:
- get
- list
- watch
- apiGroups:
- ""
resources:
- events
verbs:
- create
- patch
- apiGroups:
- "extensions"
- "networking.k8s.io"
resources:
- ingresses
verbs:
- get
- list
- watch
- apiGroups:
- "extensions"
- "networking.k8s.io"
resources:
- ingresses/status
verbs:
- update
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: Role
metadata:
name: nginx-ingress-role
namespace: ingress-nginx
labels:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
rules:
- apiGroups:
- ""
resources:
- configmaps
- pods
- secrets
- namespaces
verbs:
- get
- apiGroups:
- ""
resources:
- configmaps
resourceNames:
# Defaults to "-"
# Here: "-"
# This has to be adapted if you change either parameter
# when launching the nginx-ingress-controller.
- "ingress-controller-leader-nginx"
verbs:
- get
- update
- apiGroups:
- ""
resources:
- configmaps
verbs:
- create
- apiGroups:
- ""
resources:
- endpoints
verbs:
- get
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: RoleBinding
metadata:
name: nginx-ingress-role-nisa-binding
namespace: ingress-nginx
labels:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: Role
name: nginx-ingress-role
subjects:
- kind: ServiceAccount
name: nginx-ingress-serviceaccount
namespace: ingress-nginx
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRoleBinding
metadata:
name: nginx-ingress-clusterrole-nisa-binding
labels:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: nginx-ingress-clusterrole
subjects:
- kind: ServiceAccount
name: nginx-ingress-serviceaccount
namespace: ingress-nginx
---
apiVersion: apps/v1
kind: DaemonSet
metadata:
name: nginx-ingress-controller
namespace: ingress-nginx
labels:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
spec:
selector:
matchLabels:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
template:
metadata:
labels:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
annotations:
prometheus.io/port: "10254"
prometheus.io/scrape: "true"
spec:
hostNetwork: true
# wait up to five minutes for the drain of connections
terminationGracePeriodSeconds: 300
serviceAccountName: nginx-ingress-serviceaccount
nodeSelector:
kubernetes.io/os: linux
containers:
- name: nginx-ingress-controller
image: lizhenliang/nginx-ingress-controller:0.30.0
args:
- /nginx-ingress-controller
- --configmap=$(POD_NAMESPACE)/nginx-configuration
- --tcp-services-configmap=$(POD_NAMESPACE)/tcp-services
- --udp-services-configmap=$(POD_NAMESPACE)/udp-services
- --publish-service=$(POD_NAMESPACE)/ingress-nginx
- --annotations-prefix=nginx.ingress.kubernetes.io
securityContext:
allowPrivilegeEscalation: true
capabilities:
drop:
- ALL
add:
- NET_BIND_SERVICE
# www-data -> 101
runAsUser: 101
env:
- name: POD_NAME
valueFrom:
fieldRef:
fieldPath: metadata.name
- name: POD_NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
ports:
- name: http
containerPort: 80
protocol: TCP
- name: https
containerPort: 443
protocol: TCP
livenessProbe:
failureThreshold: 3
httpGet:
path: /healthz
port: 10254
scheme: HTTP
initialDelaySeconds: 10
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 10
readinessProbe:
failureThreshold: 3
httpGet:
path: /healthz
port: 10254
scheme: HTTP
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 10
lifecycle:
preStop:
exec:
command:
- /wait-shutdown