Kubernetes之Ingress
通过对Service的了解，暴露服务给外部可以使用NodePort和LoadBalancer两种方式，说明如下：
- NodePort会占用很多集群机器的端口，那么当集群服务变多的时候，这个缺点就愈发明显
- LoadBalancer方式的每个service需要一个LB，浪费、麻烦，并且需要kubernetes之外设备的支持
所以再此基础上，kubernetes提供了Ingress资源对象，Ingress只需要一个NodePort或者一个LB就可以满足暴露多个Service的需求。Ingress相当于一个7层的负载均衡器，是kubernetes对反向代理的一个抽象，它的工作原理类似于Nginx，可以理解成在Ingress里建立诸多映射规则，Ingress Controller通过监听这些配置规则并转化成Nginx的反向代理配置 , 然后对外部提供服务。
一、环境搭建（注意选择自己需要的版本）
我们需要这两个yaml文件，如下：
https://github.com/kubernetes/ingress-nginx/blob/nginx-0.30.0/deploy/static/mandatory.yaml
https://github.com/kubernetes/ingress-nginx/blob/nginx-0.30.0/deploy/static/provider/baremetal/service-nodeport.yaml
其中service-nodeport.yaml 做如下修改，端口可以自己指定：
文件具体内容如下：
mandatory.yaml
apiVersion: v1
kind: Namespace
metadata:
name: ingress-nginx
labels:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
---
kind: ConfigMap
apiVersion: v1
metadata:
name: nginx-configuration
namespace: ingress-nginx
labels:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
---
kind: ConfigMap
apiVersion: v1
metadata:
name: tcp-services
namespace: ingress-nginx
labels:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
---
kind: ConfigMap
apiVersion: v1
metadata:
name: udp-services
namespace: ingress-nginx
labels:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: nginx-ingress-serviceaccount
namespace: ingress-nginx
labels:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRole
metadata:
name: nginx-ingress-clusterrole
labels:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
rules:
- apiGroups:
- ""
resources:
- configmaps
- endpoints
- nodes
- pods
- secrets
verbs:
- list
- watch
- apiGroups:
- ""
resources:
- nodes
verbs:
- get
- apiGroups:
- ""
resources:
- services
verbs:
- get
- list
- watch
- apiGroups:
- ""
resources:
- events
verbs:
- create
- patch
- apiGroups:
- "extensions"
- "networking.k8s.io"
resources:
- ingresses
verbs:
- get
- list
- watch
- apiGroups:
- "extensions"
- "networking.k8s.io"
resources:
- ingresses/status
verbs:
- update
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: Role
metadata:
name: nginx-ingress-role
namespace: ingress-nginx
labels:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
rules:
- apiGroups:
- ""
resources:
- configmaps
- pods
- secrets
- namespaces
verbs:
- get
- apiGroups:
- ""
resources:
- configmaps
resourceNames:
# Defaults to "-"
# Here: "-"
# This has to be adapted if you change either parameter
# when launching the nginx-ingress-controller.
- "ingress-controller-leader-nginx"
verbs:
- get
- update
- apiGroups:
- ""
resources:
- configmaps
verbs:
- create
- apiGroups:
- ""
resources:
- endpoints
verbs:
- get
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: RoleBinding
metadata:
name: nginx-ingress-role-nisa-binding
namespace: ingress-nginx
labels:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: Role
name: nginx-ingress-role
subjects:
- kind: ServiceAccount
name: nginx-ingress-serviceaccount
namespace: ingress-nginx
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRoleBinding
metadata:
name: nginx-ingress-clusterrole-nisa-binding
labels:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: nginx-ingress-clusterrole
subjects:
- kind: ServiceAccount
name: nginx-ingress-serviceaccount
namespace: ingress-nginx
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx-ingress-controller
namespace: ingress-nginx
labels:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
spec:
replicas: 1
selector:
matchLabels:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
template:
metadata:
labels:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
annotations:
prometheus.io/port: "10254"
prometheus.io/scrape: "true"
spec:
# wait up to five minutes for the drain of connections
hostNetwork: true
terminationGracePeriodSeconds: 300
serviceAccountName: nginx-ingress-serviceaccount
nodeSelector:
kubernetes.io/os: linux
containers:
- name: nginx-ingress-controller
image: quay.io/kubernetes-ingress-controller/nginx-ingress-controller:0.30.0
args:
- /nginx-ingress-controller
- --configmap=$(POD_NAMESPACE)/nginx-configuration
- --tcp-services-configmap=$(POD_NAMESPACE)/tcp-services
- --udp-services-configmap=$(POD_NAMESPACE)/udp-services
- --publish-service=$(POD_NAMESPACE)/ingress-nginx
- --annotations-prefix=nginx.ingress.kubernetes.io
securityContext:
allowPrivilegeEscalation: true
capabilities:
drop:
- ALL
add:
- NET_BIND_SERVICE
# www-data -> 101
runAsUser: 101
env:
- name: POD_NAME
valueFrom:
fieldRef:
fieldPath: metadata.name
- name: POD_NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
ports:
- name: http
containerPort: 80
protocol: TCP
- name: https
containerPort: 443
protocol: TCP
#livenessProbe:
# failureThreshold: 3
#httpGet:
# path: /healthz
#port: 10254
#scheme: HTTP
#initialDelaySeconds: 10
#periodSeconds: 10
#successThreshold: 1
#timeoutSeconds: 10
#readinessProbe:
# failureThreshold: 3
#httpGet:
# path: /healthz
#port: 10254
#scheme: HTTP
#periodSeconds: 10
# successThreshold: 1
#timeoutSeconds: 10
lifecycle:
preStop:
exec:
command:
- /wait-shutdown
---
apiVersion: v1
kind: LimitRange
metadata:
name: ingress-nginx
namespace: ingress-nginx
labels:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
spec:
limits:
- min:
memory: 90Mi
cpu: 100m
type: Container
service-nodeport.yaml
apiVersion: v1
kind: Service
metadata:
name: ingress-nginx
namespace: ingress-nginx
labels:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
spec:
type: NodePort
ports:
- name: http
port: 80
targetPort: 80
protocol: TCP
nodePort: 32005
- name: https
port: 443
targetPort: 443
protocol: TCP
nodePort: 32006
selector:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
创建：
kubectl create -f mandatory.yaml
kubectl create -f service-nodeport.yaml
测试访问：http://ip+端口
接下来我们部署具体的应用！！
二、部署tomcat
vim ingress-tomcat.yaml
apiVersion: v1
kind: Service
metadata:
name: tomcat-svc
namespace: dev
spec:
selector:
app: tomcat
ports:
- name: tomcat-http
port: 8080
targetPort: 8080
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: tomcat-deploy
namespace: dev
spec:
replicas: 3
selector:
matchLabels:
app: tomcat
template:
metadata:
labels:
app: tomcat
spec:
containers:
- name: tomcat
image: tomcat:7-alpine
ports:
- name: httpd
containerPort: 8080
---
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: ingress-tomcat
namespace: dev
annotations:
kubernets.io/ingress.class: "nginx"
spec:
rules:
- host: mytomcat.dev.com
http:
paths:
- path:
backend:
serviceName: tomcat-svc #Service名称
servicePort: 8080
kubectl create -f ingress-tomcat.yaml
kubectl describe ingress -n dev
访问测试：http://域名+端口
注：如果域名是本地测试使用，必须在Hosts文件中配置：
配置https
[root@master ingress-new]# openssl genrsa -out tls.key 2048
Generating RSA private key, 2048 bit long modulus
..................................+++
............+++
e is 65537 (0x10001)
注：必须与自己域名保持一致
[root@master ingress-new]# openssl req -new -x509 -key tls.key -out tls.crt -subj /C=CN/ST=Beijing/L=Beijing/O=DevOps/CN=mytomcat.dev.com
注：相关名称可以自行修改
[root@master ingress-new]# kubectl create secret tls tomcat-ingress-secret --cert=tls.crt --key=tls.key
secret/tomcat-ingress-secret created
[root@master ingress-new]# kubectl get secret
NAME TYPE DATA AGE
default-token-srx5s kubernetes.io/service-account-token 3 32d
tomcat-ingress-secret kubernetes.io/tls 2 7s
[root@master ingress-new]#
修改yaml文件内容：
修改后完整文件如下：
apiVersion: v1
kind: Service
metadata:
name: tomcat-svc
namespace: dev
spec:
selector:
app: tomcat
ports:
- name: tomcat-http
port: 8080
targetPort: 8080
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: tomcat-deploy
namespace: dev
spec:
replicas: 3
selector:
matchLabels:
app: tomcat
template:
metadata:
labels:
app: tomcat
spec:
containers:
- name: tomcat
image: tomcat:7-alpine
ports:
- name: httpd
containerPort: 8080
---
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: ingress-tomcat
namespace: dev
annotations:
kubernets.io/ingress.class: "nginx"
spec:
tls:
- hosts:
- mytomcat.dev.com
secretName: tomcat-ingress-secret
rules:
- host: mytomcat.dev.com
http:
paths:
- path:
backend:
serviceName: tomcat-svc #Service名称
servicePort: 8080
创建完成之后查看信息：
kubectl get svc -n ingress-nginx
https的访问端口设置成了32006，访问看看：
三、部署自己的springboot应用
创建证书：
[root@master ingress-new]# openssl genrsa -out tls.key 2048
Generating RSA private key, 2048 bit long modulus
.......+++
.................................................................................+++
e is 65537 (0x10001)
[root@master ingress-new]# openssl req -new -x509 -key tls.key -out tls.crt -subj /C=CN/ST=Beijing/L=Beijing/O=DevOps/CN=boot.dev.com
[root@master ingress-new]# kubectl create secret tls boot-ingress-secret --cert=tls.crt --key=tls.key
secret/boot-ingress-secret created
[root@master ingress-new]# kubectl get secret
NAME TYPE DATA AGE
boot-ingress-secret kubernetes.io/tls 2 17s
default-token-srx5s kubernetes.io/service-account-token 3 34d
tomcat-ingress-secret kubernetes.io/tls 2 2d17h
[root@master ingress-new]# kubectl describe secret boot-ingress-secret
Name: boot-ingress-secret
Namespace: default
Labels: <none>
Annotations: <none>
Type: kubernetes.io/tls
Data
====
tls.key: 1679 bytes
tls.crt: 1281 bytes
vim ingress-boot.yaml
apiVersion: v1
kind: Service
metadata:
name: boot-svc
namespace: dev
spec:
selector:
app: boot
ports:
- name: boot-http
port: 8082
targetPort: 8082
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: boot-deploy
namespace: dev
spec:
replicas: 3
selector:
matchLabels:
app: boot
template:
metadata:
labels:
app: boot
spec:
containers:
- name: boot
image: registry.cn-hangzhou.aliyuncs.com/button_dev/test:v1.3
ports:
- name: httpd
containerPort: 8082
---
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: ingress-boot
namespace: dev
annotations:
kubernets.io/ingress.class: "nginx"
spec:
tls:
- hosts:
- boot.dev.com
secretName: boot-imgress-secret
rules:
- host: boot.dev.com
http:
paths:
- path:
backend:
serviceName: boot-svc #Service名称
servicePort: 8082
kubectl create -f ingress-boot.yaml
测试访问：https://boot.dev.com:32006/doc.html
查看pod中容器的运行日志：
kubectl logs -f boot-deploy-77f996f7c8-8njnz -n dev
注:如果一个pod中含有多个不同的容器，可以通过加上"-c"参数来进行查看
例如： kubectl logs -f boot-deploy-77f996f7c8-8njnz -n dev -c busybox