Ingress-nginx配置方法
一、Ingress 及 Ingress Controller 简介
Ingress简单的理解: 原先暴露的service,现在给定个统一的访问入口。
Ingress 是 k8s 资源对象，用于对外暴露服务，该资源对象定义了不同主机名（域名）及 URL 和对应后端 Service（k8s Service）的绑定，根据不同的路径路由 http 和 https 流量。而 Ingress Contoller 是一个 pod 服务，封装了一个 web 前端负载均衡器，同时在其基础上实现了动态感知 Ingress 并根据 Ingress 的定义动态生成 前端 web 负载均衡器的配置文件，比如 Nginx Ingress Controller 本质上就是一个 Nginx，只不过它能根据 Ingress 资源的定义动态生成 Nginx 的配置文件，然后动态 Reload。
二、Ingress 组成
- 将Nginx的配置抽象成一个Ingress对象，每添加一个新的服务只需写一个新的Ingress的yaml文件即可
- 将新加入的Ingress转化成Nginx的配置文件并使之生效
- ingress controller
- ingress服务
三、ingress的工作原理
ingress具体的工作原理如下:
ingress contronler通过与k8s的api进行交互，动态的去感知k8s集群中ingress服务规则的变化，然后读取它，并按照定义的ingress规则，转发到k8s集群中对应的service。
而这个ingress规则写明了哪个域名对应k8s集群中的哪个service，然后再根据ingress-controller中的nginx配置模板，生成一段对应的nginx配置。
然后再把该配置动态的写到ingress-controller的pod里，该ingress-controller的pod里面运行着一个nginx服务，控制器会把生成的nginx配置写入到nginx的配置文件中，然后reload一下，使其配置生效。以此来达到域名分配置及动态更新的效果。
四、Ingress 可以解决什么问题？
动态配置服务
如果按照传统方式, 当新增加一个服务时, 我们可能需要在流量入口加一个反向代理指向我们新的k8s服务. 而如果用了Ingress, 只需要配置好这个服务, 当服务启动时, 会自动注册到Ingress的中, 不需要而外的操作.
减少不必要的暴露端口
配置过k8s的都清楚, 第一步是要关闭防火墙的, 主要原因是k8s的很多服务会以NodePort方式映射出去, 这样就相当于给宿主机打了很多孔, 既不安全也不优雅. 而Ingress可以避免这个问题, 除了Ingress自身服务可能需要映射出去, 其他服务都不要用NodePort方式
五、Ingress-nginx配置示例
1、部署httpd服务
命名空间
[root@k8s-master httpd]# cat httpd-namespace.yml
apiVersion: v1
kind: Namespace
metadata:
name: lzy-ns
labels:
name: lzy-ns
Deployment资源
[root@k8s-master httpd]# cat httpd-deployment.yml
apiVersion: apps/v1
kind: Deployment
metadata:
name: httpd-deploy
namespace: lzy-ns
spec:
replicas: 3
selector:
matchLabels:
app: lzy-ns
template:
metadata:
labels:
app: lzy-ns
spec:
containers:
- name: httpd
image: httpd
service端口暴露
[root@k8s-master httpd]# cat httpd-service.yml
apiVersion: v1
kind: Service
metadata:
name: httpd-svc
namespace: lzy-ns
spec:
type: NodePort
selector:
app: lzy-ns
ports:
- name: http-port
port: 80
targetPort: 80
nodePort: 31033
启动httpd程序
[root@k8s-master httpd]# kubectl apply -f httpd-namespace.yml
[root@k8s-master httpd]# kubectl apply -f httpd-namespace.yml
[root@k8s-master httpd]# kubectl apply -f httpd-service.yml
查看httpd程序启动情况
[root@k8s-master httpd]# kubectl get all -n lzy-ns
NAME READY STATUS RESTARTS AGE
pod/httpd-deploy-6cdf8d7fcd-hmprv 1/1 Running 0 30s
pod/httpd-deploy-6cdf8d7fcd-qpwsj 1/1 Running 0 30s
pod/httpd-deploy-6cdf8d7fcd-znzft 1/1 Running 0 30s
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/httpd-svc NodePort 10.106.248.82 80:31033/TCP 32s
NAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/httpd-deploy 3/3 3 3 30s
NAME DESIRED CURRENT READY AGE
replicaset.apps/httpd-deploy-6cdf8d7fcd 3 3 3 30s
2、部署tomcat服务
命名空间
[root@k8s-master tomcat]# cat tomcat-nmaespace.yml
apiVersion: v1
kind: Namespace
metadata:
name: lzy-ns
labels:
name: lzy-ns
Deployment资源
[root@k8s-master tomcat]# cat tomcat-deployment.yml
apiVersion: apps/v1
kind: Deployment
metadata:
name: tomcat-deploy
namespace: lzy-ns
spec:
replicas: 1
selector:
matchLabels:
app: lzy-tomcat
template:
metadata:
labels:
app: lzy-tomcat
spec:
containers:
- name: tomcat
image: tomcat:8.5.45
imagePullPolicy: IfNotPresent
service端口暴露
[root@k8s-master tomcat]# cat tomcat-service.yml
apiVersion: v1
kind: Service
metadata:
name: tomcat-svc
namespace: lzy-ns
spec:
type: NodePort
selector:
app: lzy-tomcat
ports:
- name: tomcat-port
port: 8080
targetPort: 8080
nodePort: 32033
启动程序
[root@k8s-master tomcat]# kubectl apply -f tomcat-nmaespace.yml
[root@k8s-master tomcat]# kubectl apply -f tomcat-deployment.yml
[root@k8s-master tomcat]# kubectl apply -f tomcat-service.yml
查看tomcat程序启动情况
[root@k8s-master tomcat]# kubectl get all -n lzy-ns
NAME READY STATUS RESTARTS AGE
pod/httpd-deploy-6cdf8d7fcd-hmprv 1/1 Running 0 9m30s
pod/httpd-deploy-6cdf8d7fcd-qpwsj 1/1 Running 0 9m30s
pod/httpd-deploy-6cdf8d7fcd-znzft 1/1 Running 0 9m30s
pod/tomcat-deploy-797756cb97-2mxr6 1/1 Running 0 96s
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/httpd-svc NodePort 10.106.248.82 80:31033/TCP 9m32s
service/tomcat-svc NodePort 10.100.147.5 8080:32033/TCP 93s
NAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/httpd-deploy 3/3 3 3 9m30s
deployment.apps/tomcat-deploy 1/1 1 1 96s
NAME DESIRED CURRENT READY AGE
replicaset.apps/httpd-deploy-6cdf8d7fcd 3 3 3 9m30s
replicaset.apps/tomcat-deploy-797756cb97 1 1 1 96s
3、部署Ingress服务
Ingress:
- (1)
Ingress controller:
将新加入的Ingress转化为反向代理服务器的配置文件，并使之生效。 - (2)
Ingress :
将反向代理服务器的配置抽象成一个Ingress对象，每添加一个新的服务，只需要写一个新的Ingress的yaml文件即可。
Nginx :反向代理服务器。
需要解决了两个问题:
- 动态的配置服务。
- 减少不必要的暴露端口。
基于nginx的ingress controller根据不同的开发公司，又分为两种:
- k8s社区版的: Ingerss - nginx.
- nginx公司自己开发的: nginx- ingress .
- 在gitbub上找到所需的ingress的yaml文件
部署Ingress服务
deployment资源
[root@k8s-master nginx]# vim Ingress-deployment.yaml
apiVersion: v1
kind: Namespace
metadata:
name: ingress-nginx
labels:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
---
kind: ConfigMap
apiVersion: v1
metadata:
name: nginx-configuration
namespace: ingress-nginx
labels:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
---
kind: ConfigMap
apiVersion: v1
metadata:
name: tcp-services
namespace: ingress-nginx
labels:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
---
kind: ConfigMap
apiVersion: v1
metadata:
name: udp-services
namespace: ingress-nginx
labels:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: nginx-ingress-serviceaccount
namespace: ingress-nginx
labels:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRole
metadata:
name: nginx-ingress-clusterrole
labels:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
rules:
- apiGroups:
- ""
resources:
- configmaps
- endpoints
- nodes
- pods
- secrets
verbs:
- list
- watch
- apiGroups:
- ""
resources:
- nodes
verbs:
- get
- apiGroups:
- ""
resources:
- services
verbs:
- get
- list
- watch
- apiGroups:
- ""
resources:
- events
verbs:
- create
- patch
- apiGroups:
- "extensions"
- "networking.k8s.io"
resources:
- ingresses
verbs:
- get
- list
- watch
- apiGroups:
- "extensions"
- "networking.k8s.io"
resources:
- ingresses/status
verbs:
- update
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: Role
metadata:
name: nginx-ingress-role
namespace: ingress-nginx
labels:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
rules:
- apiGroups:
- ""
resources:
- configmaps
- pods
- secrets
- namespaces
verbs:
- get
- apiGroups:
- ""
resources:
- configmaps
resourceNames:
# Defaults to "-"
# Here: "-"
# This has to be adapted if you change either parameter
# when launching the nginx-ingress-controller.
- "ingress-controller-leader-nginx"
verbs:
- get
- update
- apiGroups:
- ""
resources:
- configmaps
verbs:
- create
- apiGroups:
- ""
resources:
- endpoints
verbs:
- get
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: RoleBinding
metadata:
name: nginx-ingress-role-nisa-binding
namespace: ingress-nginx
labels:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: Role
name: nginx-ingress-role
subjects:
- kind: ServiceAccount
name: nginx-ingress-serviceaccount
namespace: ingress-nginx
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRoleBinding
metadata:
name: nginx-ingress-clusterrole-nisa-binding
labels:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: nginx-ingress-clusterrole
subjects:
- kind: ServiceAccount
name: nginx-ingress-serviceaccount
namespace: ingress-nginx
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx-ingress-controller
namespace: ingress-nginx
labels:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
spec:
replicas: 1
selector:
matchLabels:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
template:
metadata:
labels:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
annotations:
prometheus.io/port: "10254"
prometheus.io/scrape: "true"
spec:
hostNetwork: true
# wait up to five minutes for the drain of connections
terminationGracePeriodSeconds: 300
serviceAccountName: nginx-ingress-serviceaccount
nodeSelector:
kubernetes.io/os: linux
containers:
- name: nginx-ingress-controller
image: quay.io/kubernetes-ingress-controller/nginx-ingress-controller:0.29.0
args:
- /nginx-ingress-controller
- --configmap=$(POD_NAMESPACE)/nginx-configuration
- --tcp-services-configmap=$(POD_NAMESPACE)/tcp-services
- --udp-services-configmap=$(POD_NAMESPACE)/udp-services
- --publish-service=$(POD_NAMESPACE)/ingress-nginx
- --annotations-prefix=nginx.ingress.kubernetes.io
securityContext:
allowPrivilegeEscalation: true
capabilities:
drop:
- ALL
add:
- NET_BIND_SERVICE
# www-data -> 101
runAsUser: 101
env:
- name: POD_NAME
valueFrom:
fieldRef:
fieldPath: metadata.name
- name: POD_NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
ports:
- name: http
containerPort: 80
protocol: TCP
- name: https
containerPort: 443
protocol: TCP
livenessProbe:
failureThreshold: 3
httpGet:
path: /healthz
port: 10254
scheme: HTTP
initialDelaySeconds: 10
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 10
readinessProbe:
failureThreshold: 3
httpGet:
path: /healthz
port: 10254
scheme: HTTP
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 10
lifecycle:
preStop:
exec:
command:
- /wait-shutdown
---
apiVersion: v1
kind: LimitRange
metadata:
name: ingress-nginx
namespace: ingress-nginx
labels:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
spec:
limits:
- min:
memory: 90Mi
cpu: 100m
type: Container
service端口暴露
[root@k8s-master nginx]# vim Ingress-service.yaml
apiVersion: v1
kind: Service
metadata:
name: ingress-nginx
namespace: ingress-nginx
spec:
type: NodePort
ports:
- name: httpd
port: 80
targetPort: 80
- name: https
port: 443
selector:
app: ingress-nginx
启动程序
[root@k8s-master nginx]# kubectl apply -f Ingress-deployment.yaml
[root@k8s-master nginx]# kubectl apply -f Ingress-service.yaml
查看程序启动情况
[root@k8s-master nginx]# kubectl get svc -n ingress-nginx
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
ingress-nginx NodePort 10.96.94.210 80:30023/TCP,443:30301/TCP 4h10m
创建Ingress资源
ingress ：
- ingress-nginx-controller:
动态感知ingress 资源的变化 - ingress:
创建svc5ingress-nginx-contr011er 关联的规则
编写ingress的yaml文件
[root@k8s-master nginx]# vim ingress.yaml
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: lzy-ingress
namespace: lzy-ns
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /
spec:
rules: #规则
- host: ingress.lzy.com #域名
http:
paths:
- path: /
backend:
serviceName: httpd-svc #关联service
servicePort: 80 #关联service的映射端口
- path: /tomcat
backend:
serviceName: tomcat-svc #关联service
servicePort: 8080 #关联service的映射端口
启用Ingress
[root@k8s-master nginx]# kubectl apply -f ingress.yaml
查看启用情况
[root@k8s-master nginx]# kubectl get pod -n ingress-nginx -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
nginx-ingress-controller-6889cffb4d-h7qf2 1/1 Running 0 4h20m 192.168.1.221 k8s-master
[root@k8s-master nginx]# kubectl get ingresses. -n lzy-ns
NAME CLASS HOSTS ADDRESS PORTS AGE
lzy-ingress ingress.lzy.com 80 6s
[root@k8s-master nginx]# kubectl describe ingresses. -n lzy-ns
Name: lzy-ingress
Namespace: lzy-ns
Address: 10.96.94.210
Default backend: default-http-backend:80 ()
Rules:
Host Path Backends
---- ---- --------
ingress.lzy.com
/ httpd-svc:80 (10.244.0.21:80,10.244.0.22:80,10.244.0.23:80) ##重点
/tomcat tomcat-svc:8080 (10.244.0.24:8080) ##重点
Annotations: nginx.ingress.kubernetes.io/rewrite-target: /
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal CREATE 40s nginx-ingress-controller Ingress lzy-ns/lzy-ingress
Normal UPDATE 19s nginx-ingress-controller Ingress lzy-ns/lzy-ingress
测试是否正常
现做下本地host绑定域名
192.168.1.221 ingress.lzy.com
httpd:
tomcat:
完成！！！！