Linux企业实战之容器（十一）——Kubernetes（6）
Ingress
- 一种全局的、为了代理不同后端 Service 而设置的负载均衡服务，就是 Kubernetes 里的Ingress 服务；
- Ingress由两部分组成：Ingress controller和Ingress服务；
- Ingress Controller 会根据你定义的 Ingress 对象，提供对应的代理能力。业界常用的各种反向代理项目，比如 Nginx、HAProxy、Envoy、Traefik 等，都已经为Kubernetes 专门维护了对应的 Ingress Controller
（1）Ingress Controller
-
部署ingress-controller
步骤一：拉取镜像，下载资源清单文件
docker pull quay.io/kubernetes-ingress-controller/nginx-ingress-controller:0.30.0
在server1上下载资源清单文件：
wget https://raw.githubusercontent.com/kubernetes/ingress-nginx/nginx-0.30.0/deploy/static/mandatory.yaml
步骤二：利用下载好的资源清单文件来创建pod，和namespace
kubectl apply -f mandatory.yaml
kubectl get namespace
kubectl -n ingress-nginx get pod
kubectl -n ingress-nginx logs nginx-ingress-controller-58956cf6ff-tvhv7 #查看这个pod的日志
步骤三：部署ingress-nginx这个service
wget https://raw.githubusercontent.com/kubernetes/ingress-nginx/nginx-0.30.0/deploy/static/provider/baremetal/service-nodeport.yaml #下载资源清单
cat service-nodeport.yaml
kubectl apply -f service-nodeport.yaml
kubectl -n ingress-nginx get svc #查看创建的service
kubectl -n ingress-nginx describe svc ingress-nginx #查看创建的service的详细信息
步骤四：创建一个ingress服务，让其来调度我们之前创建的pod
vim ingress.yaml
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
name: ingress-demo
spec:
backend:
serviceName: myservice
servicePort: 80
kubectl apply -f ingress.yaml
kubectl get ingress
步骤五：通过pod的IP加我们的ingress-nginx的port来访问，我们之前创建的pod
kubectl get pod -o wide
kubectl -n ingress-nginx get svc
步骤六：修改资源清单文件，让外部可以通过域名来访问pod
vim ingress.yaml
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
name: ingress-demo
spec:
backend:
serviceName: myservice
servicePort: 80
rules:
- host: www1.westos.org
http:
paths:
- path: /
backend:
serviceName: myservice
servicePort: 80
kubectl apply -f ingress.yaml
步骤七：在外部主机上添加解析，然后通过域名加端口的方式来访问pod
vim /etc/hosts
192.168.43.102 www1.westos.org
步骤八：在创建一个通过域名来访问的pod，也是被我们的ingress服务来调度，给其编辑一个新的域名，这样我们就可以访问不同的域名来访问不同的pod
vim ingress1.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: myapp-deployment
labels:
app: myapp
spec:
replicas: 2
selector:
matchLabels:
app: myapp
template:
metadata:
labels:
app: myapp
spec:
containers:
- name: myapp
image: reg.westos.org:5000/myapp:v2
ports:
- containerPort: 80
---
kind: Service
apiVersion: v1
metadata:
name: mynginx
spec:
ports:
- protocol: TCP
port: 80
targetPort: 80
selector:
app: myapp
---
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
name: ingress-example
spec:
rules:
- host: www2.westos.org
http:
paths:
- path: /
backend:
serviceName: mynginx
servicePort: 80
kubectl apply -f ingress1.yaml
kubectl get ingress
步骤九：在外部主机上添加解析，然后通过域名加端口的方式来访问pod
vim /etc/hosts
192.168.43.102 www2.westos.org
也可以通过curl命令来进行访问，这里我们可以看到，通过同一个ingress控制器，来调度两个不同service下的pod，且通过外部主机做解析后，我们可以域名来进行访问不同的pod
步骤十：也可以通过同一域名不同路径的方法，来访问不同的service下的pod
vim ingress2.yaml
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
name: simple-fanout-example
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /
spec:
rules:
- host: www3.westos.org
http:
paths:
- path: /v1
backend:
serviceName: myservice
servicePort: 80
- path: /v2
backend:
serviceName: mynginx
servicePort: 80
kubectl apply -f ingress2.yaml
kubectl describ ingress simple-fanout-example
步骤十一：在外部主机上添加解析，然后通过域名加端口再加路径的方式来访问pod
（2）DaemonSet结合nodeselector来部署ingress-controller到特定的node
-
用DaemonSet结合nodeselector来部署ingress-controller到特定的node上，然后使用HostNetwork直接把该pod与宿主机node的网络打通，直接使用宿主机的80/433端口就能访问服务。
- 优点是整个请求链路最简单，性能相对NodePort模式更好。
- 缺点是由于直接利用宿主机节点的网络和端口，一个node只能部署一个ingress-controller pod。
- 比较适合大并发的生产环境使用。
步骤一：修改mandatory.yaml 文件，将service的类型修改为daemonset，添加HostName网络模式
kubectl get nodes --show-labels #查看各个node的标签
kubectl label nodes server2 ingress=nginx #设置ingress controller节点的标签
vim mandatory.yaml
apiVersion: v1
kind: Namespace
metadata:
name: ingress-nginx
labels:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
---
kind: ConfigMap
apiVersion: v1
metadata:
name: nginx-configuration
namespace: ingress-nginx
labels:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
---
kind: ConfigMap
apiVersion: v1
metadata:
name: tcp-services
namespace: ingress-nginx
labels:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
---
kind: ConfigMap
apiVersion: v1
metadata:
name: udp-services
namespace: ingress-nginx
labels:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: nginx-ingress-serviceaccount
namespace: ingress-nginx
labels:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRole
metadata:
name: nginx-ingress-clusterrole
labels:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
rules:
- apiGroups:
- ""
resources:
- configmaps
- endpoints
- nodes
- pods
- secrets
verbs:
- list
- watch
- apiGroups:
- ""
resources:
- nodes
verbs:
- get
- apiGroups:
- ""
resources:
- services
verbs:
- get
- list
- watch
- apiGroups:
- ""
resources:
- events
verbs:
- create
- patch
- apiGroups:
- "extensions"
- "networking.k8s.io"
resources:
- ingresses
verbs:
- get
- list
- watch
- apiGroups:
- "extensions"
- "networking.k8s.io"
resources:
- ingresses/status
verbs:
- update
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: Role
metadata:
name: nginx-ingress-role
namespace: ingress-nginx
labels:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
rules:
- apiGroups:
- ""
resources:
- configmaps
- pods
- secrets
- namespaces
verbs:
- get
- apiGroups:
- ""
resources:
- configmaps
resourceNames:
# Defaults to "-"
# Here: "-"
# This has to be adapted if you change either parameter
# when launching the nginx-ingress-controller.
- "ingress-controller-leader-nginx"
verbs:
- get
- update
- apiGroups:
- ""
resources:
- configmaps
verbs:
- create
- apiGroups:
- ""
resources:
- endpoints
verbs:
- get
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: RoleBinding
metadata:
name: nginx-ingress-role-nisa-binding
namespace: ingress-nginx
labels:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: Role
name: nginx-ingress-role
subjects:
- kind: ServiceAccount
name: nginx-ingress-serviceaccount
namespace: ingress-nginx
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRoleBinding
metadata:
name: nginx-ingress-clusterrole-nisa-binding
labels:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: nginx-ingress-clusterrole
subjects:
- kind: ServiceAccount
name: nginx-ingress-serviceaccount
namespace: ingress-nginx
---
apiVersion: apps/v1
kind: DaemonSet #修改service的类型为DaemonSet
metadata:
name: nginx-ingress-controller
namespace: ingress-nginx
labels:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
spec:
#replicas: 1
selector:
matchLabels:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
template:
metadata:
labels:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
annotations:
prometheus.io/port: "10254"
prometheus.io/scrape: "true"
spec:
hostNetwork: true #添加网络模式
# wait up to five minutes for the drain of connections
terminationGracePeriodSeconds: 300
serviceAccountName: nginx-ingress-serviceaccount
nodeSelector:
ingress: nginx #修改标签
containers:
- name: nginx-ingress-controller
image: reg.westos.org:5000/nginx-ingress-controller
args:
- /nginx-ingress-controller
- --configmap=$(POD_NAMESPACE)/nginx-configuration
- --tcp-services-configmap=$(POD_NAMESPACE)/tcp-services
- --udp-services-configmap=$(POD_NAMESPACE)/udp-services
- --publish-service=$(POD_NAMESPACE)/ingress-nginx
- --annotations-prefix=nginx.ingress.kubernetes.io
securityContext:
allowPrivilegeEscalation: true
capabilities:
drop:
- ALL
add:
- NET_BIND_SERVICE
# www-data -> 101
runAsUser: 101
env:
- name: POD_NAME
valueFrom:
fieldRef:
fieldPath: metadata.name
- name: POD_NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
ports:
- name: http
containerPort: 80
protocol: TCP
- name: https
containerPort: 443
protocol: TCP
livenessProbe:
failureThreshold: 3
httpGet:
path: /healthz
port: 10254
scheme: HTTP
initialDelaySeconds: 10
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 10
readinessProbe:
failureThreshold: 3
httpGet:
path: /healthz
port: 10254
scheme: HTTP
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 10
lifecycle:
preStop:
exec:
command:
- /wait-shutdown
---
apiVersion: v1
kind: LimitRange
metadata:
name: ingress-nginx
namespace: ingress-nginx
labels:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
spec:
limits:
- min:
memory: 90Mi
cpu: 100m
type: Container
kubectl -n ingress-nginx get deployments.apps
kubectl -n ingress-nginx delete deployments.apps nginx-ingress-controller #删除之前创建的pod
kubectl apply -f mandatory.yaml
kubectl -n ingress-nginx get daemonsets.apps
kubectl -n ingress-nginx get pod
步骤二：在server2上查看端口的情况
netstat -antlp
netstat -antlp | grep 80
netstat -antlp | grep 443
步骤三：测试：直接通过域名的方式来访问pod，不用添加端口
（3）ingress TLS配置
步骤一：创建证书，并将其保存到secret中
mkdir ingress
mkdir certs
cd certs
openssl req -x509 -sha256 -nodes -days 365 -newkey rsa:2048 -keyout tls.key -out tls.crt -subj "/CN=nginxsvc/O=nginxsvc"
kubectl create secret tls tls-secret --key tls.key --cert tls.crt
kubectl get secrets
kubectl describe secrets tls-secret
步骤二：编辑ingress.yaml文件，让其生成pod时，对应的域名添加TLS加密
vim ingress.yaml
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
name: ingress-demo
spec:
tls:
- hosts:
- www1.westos.org
secretName: tls-secret
rules:
- host: www1.westos.org
http:
paths:
- path: /
backend:
serviceName: myservice
servicePort: 80
kubectl apply -f ingress.yaml
kubectl describe ingress ingress-demo
步骤三：从集群外部访问加密的pod
步骤四：添加认证
sudo yum install httpd-tools -y #安装认证工具
htpasswd -c auth bdkl #创建认证文件
cat auth #查看创建好的认证文件内容
kubectl create secret generic basic-auth --from-file=auth #将auth忍着认证人间添加到secret中
kubectl describe secrets basic-auth
- 在ingress.yaml文件中添加认证的内容
vim ingress.yaml
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
name: ingress-demo
annotations:
# type of authentication
nginx.ingress.kubernetes.io/auth-type: basic
# name of the secret that contains the user/password definitions
nginx.ingress.kubernetes.io/auth-secret: basic-auth
# message to display with an appropriate context why the authentication is required
nginx.ingress.kubernetes.io/auth-realm: 'Authentication Required - bdkl'
spec:
tls:
- hosts:
- www1.westos.org
secretName: tls-secret
rules:
- host: www1.westos.org
http:
paths:
- path: /
backend:
serviceName: myservice
servicePort: 80
kubectl apply -f ingress.yaml
kubectl describe ingress ingress-demo
- 测试，在浏览器中访问www1.westos.org
（4）Rewrite
-
annotations参数
- 将www1.westos.org的访问界面重定向为 www1.westos.org/hostname.html
步骤一：删除之前创建的ingress-demo，然后编辑资源清单文件先不添加重写规则，查看一下访问www1.westos.org的效果
kubectl delete -f ingress.yaml
vim ingress2.yaml
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
name: rewrite-example
#annotations:
# nginx.ingress.kubernetes.io/app-root: /hostname.html
spec:
rules:
- host: www1.westos.org
http:
paths:
- path: /
backend:
serviceName: myservice
servicePort: 80
kubectl apply -f ingress2.yaml
- 在集群外部的主机浏览器中访问www1.westos.org
步骤二：编辑资源清单文件ingress2.yaml 使重写规则生效
vim ingress2.yaml
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
name: rewrite-example
annotations:
nginx.ingress.kubernetes.io/app-root: /hostname.html
spec:
rules:
- host: www1.westos.org
http:
paths:
- path: /
backend:
serviceName: myservice
servicePort: 80
kubectl apply -f ingress2.yaml
-
再次进行测试，在浏览器中访问www1.westos.org
-
同样也可以通过curl命令来进行访问
curl www1.westos.org
curl www1.westos.org -L #支持重定向访问
curl -v www1.westos.org -L #查看访问的过程，并且支持重定向
