十四、kubernetes 核心技术-Ingress

1. Ingress 介绍

Ingress能把Service（Kubernetes的服务）配置成外网能够访问的URL，流量负载均衡，终止SSL，提供于域名访问的虚拟主机等，用户通过访问URL（API资源服务的形式，例如：caas.one/kibana）进入和请求Service，一个Ingress控制器负责处理所有Ingress的请求流量，它通常是一个负载均衡器，它也可以设置在边界路由器上，或者由额外的前端来帮助处理HA方式的流量。

Kubernetes暴露服务的方式目前只有三种：

1.LoadBlancer Service 2.NodePort Service 3.Ingress

Ingress 三大组件

1.反向代理负载均衡器 2.Ingress Controller 3.Ingress

反向代理负载均衡器 反向代理负载均衡器很简单，说白了就是nginx、apche等；在集群中反向代理负载均衡器可以自由部署，可以使用Replication Controller、Deployment、DaemonSet等等

Ingress Controller Ingress Controller实质上可以理解为是个监视器,Ingress Controller通过不断地跟Kubernetes API打交道,实时的感知后端Service、Pod等变化,比如新增和减少Pod,Service增加与减少等；当得到这些变化信息后，Ingress Controller在结合下文的Ingress生成配置，然后更新反向代理负载均衡器，并刷新其配置,达到服务发现的作用。

总结：Ingress解决的是新的服务加入后，域名和服务的对应问题，基本上是一个ingress的对象，通过yaml进行创建和更新进行加载。

Ingress Ingress简单理解就是个规则定义；比如某个域名对应某个Serivce,即当某个域名的请求进来时转发给某个Service；这个规则将与Ingress Controller结合，然后Ingress Controller将其动态写入到负载均衡器中，从而实现整体的服务发现和负载均衡 总结：Ingress解决的是新的服务加入后，域名和服务的对应问题，基本上是一个ingress的对象，通过yaml进行创建和更新进行加载。

• Ingress和Pod关系

  • pod和Ingress通过service关联，Ingress作为一个统一入口通过域名，由service关联一组Pod

• Ingress工作流程

2. Ingress Install

我们知道，前端的Nginx最重要负载到后端的Service上，那么如果service上没有pod该如何解决？ 官方给出的建议是部署一个默认后端，对于未知请求全部负载到这个默认后端上，这个后端不提供其它服务，只返回404.它还提供了一个http检测功能，检测nginx-ingress-controll健康状态的，通过每隔一定时间访问nginx-ingress-controll的/healthz页面，如是没有响应就 返回404之类的错误码。
通过以下yaml文件进行安装

cat > ingress-controller.yaml <<-EOF
apiVersion: v1
kind: Namespace
metadata:
  name: ingress-nginx
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx

---

kind: ConfigMap
apiVersion: v1
metadata:
  name: nginx-configuration
  namespace: ingress-nginx
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx

---
kind: ConfigMap
apiVersion: v1
metadata:
  name: tcp-services
  namespace: ingress-nginx
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx

---
kind: ConfigMap
apiVersion: v1
metadata:
  name: udp-services
  namespace: ingress-nginx
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx

---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: nginx-ingress-serviceaccount
  namespace: ingress-nginx
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx

---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRole
metadata:
  name: nginx-ingress-clusterrole
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
rules:
  - apiGroups:
      - ""
    resources:
      - configmaps
      - endpoints
      - nodes
      - pods
      - secrets
    verbs:
      - list
      - watch
  - apiGroups:
      - ""
    resources:
      - nodes
    verbs:
      - get
  - apiGroups:
      - ""
    resources:
      - services
    verbs:
      - get
      - list
      - watch
  - apiGroups:
      - ""
    resources:
      - events
    verbs:
      - create
      - patch
  - apiGroups:
      - "extensions"
      - "networking.k8s.io"
    resources:
      - ingresses
    verbs:
      - get
      - list
      - watch
  - apiGroups:
      - "extensions"
      - "networking.k8s.io"
    resources:
      - ingresses/status
    verbs:
      - update

---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: Role
metadata:
  name: nginx-ingress-role
  namespace: ingress-nginx
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
rules:
  - apiGroups:
      - ""
    resources:
      - configmaps
      - pods
      - secrets
      - namespaces
    verbs:
      - get
  - apiGroups:
      - ""
    resources:
      - configmaps
    resourceNames:
      # Defaults to "-"
      # Here: "-"
      # This has to be adapted if you change either parameter
      # when launching the nginx-ingress-controller.
      - "ingress-controller-leader-nginx"
    verbs:
      - get
      - update
  - apiGroups:
      - ""
    resources:
      - configmaps
    verbs:
      - create
  - apiGroups:
      - ""
    resources:
      - endpoints
    verbs:
      - get

---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: RoleBinding
metadata:
  name: nginx-ingress-role-nisa-binding
  namespace: ingress-nginx
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: nginx-ingress-role
subjects:
  - kind: ServiceAccount
    name: nginx-ingress-serviceaccount
    namespace: ingress-nginx

---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRoleBinding
metadata:
  name: nginx-ingress-clusterrole-nisa-binding
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: nginx-ingress-clusterrole
subjects:
  - kind: ServiceAccount
    name: nginx-ingress-serviceaccount
    namespace: ingress-nginx

---

apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: nginx-ingress-controller
  namespace: ingress-nginx
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
spec:
  selector:
    matchLabels:
      app.kubernetes.io/name: ingress-nginx
      app.kubernetes.io/part-of: ingress-nginx
  template:
    metadata:
      labels:
        app.kubernetes.io/name: ingress-nginx
        app.kubernetes.io/part-of: ingress-nginx
      annotations:
        prometheus.io/port: "10254"
        prometheus.io/scrape: "true"
    spec:
      hostNetwork: true
      # wait up to five minutes for the drain of connections
      terminationGracePeriodSeconds: 300
      serviceAccountName: nginx-ingress-serviceaccount
      nodeSelector:
        kubernetes.io/os: linux
      containers:
        - name: nginx-ingress-controller
          image: lizhenliang/nginx-ingress-controller:0.30.0
          args:
            - /nginx-ingress-controller
            - --configmap=$(POD_NAMESPACE)/nginx-configuration
            - --tcp-services-configmap=$(POD_NAMESPACE)/tcp-services
            - --udp-services-configmap=$(POD_NAMESPACE)/udp-services
            - --publish-service=$(POD_NAMESPACE)/ingress-nginx
            - --annotations-prefix=nginx.ingress.kubernetes.io
          securityContext:
            allowPrivilegeEscalation: true
            capabilities:
              drop:
                - ALL
              add:
                - NET_BIND_SERVICE
            # www-data -> 101
            runAsUser: 101
          env:
            - name: POD_NAME
              valueFrom:
                fieldRef:
                  fieldPath: metadata.name
            - name: POD_NAMESPACE
              valueFrom:
                fieldRef:
                  fieldPath: metadata.namespace
          ports:
            - name: http
              containerPort: 80
              protocol: TCP
            - name: https
              containerPort: 443
              protocol: TCP
          livenessProbe:
            failureThreshold: 3
            httpGet:
              path: /healthz
              port: 10254
              scheme: HTTP
            initialDelaySeconds: 10
            periodSeconds: 10
            successThreshold: 1
            timeoutSeconds: 10
          readinessProbe:
            failureThreshold: 3
            httpGet:
              path: /healthz
              port: 10254
              scheme: HTTP
            periodSeconds: 10
            successThreshold: 1
            timeoutSeconds: 10
          lifecycle:
            preStop:
              exec:
                command:
                  - /wait-shutdown

---

apiVersion: v1
kind: LimitRange
metadata:
  name: ingress-nginx
  namespace: ingress-nginx
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
spec:
  limits:
  - min:
      memory: 90Mi
      cpu: 100m
    type: Container
EOF

执行并查看

kubectl apply -f ingress-controller.yaml

验证配置

在浏览器访问nodeIP:80 或https:nodeIP将得到 404 NotFound 错误页面

3. 安装nginx进行测试

3.1 ingress 80端口实例

创建应用pod、service

kubectl create deplyment web01 --image=nginx:1.14
kubectl expose deployment web02  --port=80 --protocol=TCP

关联ingress 80端口

apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  name: web01-ingress
spec:
  rules:
  - host: nginx.didi.cn			# 域名
    http:
      paths:
      - backend:
          serviceName: web01			# 后端service名称
          servicePort: 80					# 后端service端口

创建并查看

kubectl apply *.yaml

host劫持

3.2 Ingress 443 端口实例

创建应用pod、service

kubectl create deplyment web01 --image=nginx:1.14
kubectl expose deployment web02  --port=80 --protocol=TCP

ca证书创建

openssl genrsa -idea -out server.key 2048
openssl req -days 46500 -x509 -sha256 -nodes -newkey rsa:2048 -keyout server.key -out server.crt

通过secret进行ca证书绑定

https需要用到ca证书，将ca证书绑定到secret资源中

apiVersion: v1
kind: Secret
metadata:
  name: didi.cn
type: kubernetes.io/tls
data:
  tls.crt: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSURXVENDQWtHZ0F3SUJBZ0lKQUpHcUlZWlVOWURKTUEwR0NTcUdTSWIzRFFFQkN3VUFNRUl4Q3pBSkJnTlYKQkFZVEFsaFlNUlV3RXdZRFZRUUhEQXhFWldaaGRXeDBJRU5wZEhreEhEQWFCZ05WQkFvTUUwUmxabUYxYkhRZwpRMjl0Y0dGdWVTQk1kR1F3SUJjTk1qRXdOVEUwTURreU16TTVXaGdQTWpFME9EQTVNRFV3T1RJek16bGFNRUl4CkN6QUpCZ05WQkFZVEFsaFlNUlV3RXdZRFZRUUhEQXhFWldaaGRXeDBJRU5wZEhreEhEQWFCZ05WQkFvTUUwUmwKWm1GMWJIUWdRMjl0Y0dGdWVTQk1kR1F3Z2dFaU1BMEdDU3FHU0liM0RRRUJBUVVBQTRJQkR3QXdnZ0VLQW9JQgpBUUMxMEZyeGF1YW82M1Q4QnhNVE9MbXdmTnZDU2w4cThQNVlWM3RqSE1YekQvdDRhdi8wN2lSN2VKNGdNUDdWCnd1MHVicGFxbkR0djF6eVVOQlJPWHpjN253L0ZzLzhPUVpxT2dTU1dTTlREdU1zMkdlVFlYWVhoaGFzTHRIS2EKZnR6RkdvcStqYnNCN2RXTExURkMydlFmd3VZTGZJVDVDNDNvKzcvck1IZDVHTEZIYkYyOS9MS2NsdnZpMEJvMApWYmdoQUd0aXE3aS9BS2diU3lQWktIMTZRU1c0S3hWZ2tCK0ZkM2ZmcjlJTGdtUkdrbG5aaHpUTkJBWFNvN3YzCmE1b1NjY2srMnhRM0NHZkhhSFIrMzk0OERVVFNNWndhdFVqV1NYd1FrNEhIbnJDMmxHS2FGcXA1YTdCMzNRbk4KQTJXMUwwZUs0SW5RQmd6dTVqclJMWkZKQWdNQkFBR2pVREJPTUIwR0ExVWREZ1FXQkJSTVBHWEowSDVqeUMxUApycGNwU2xkOEV5NXhiekFmQmdOVkhTTUVHREFXZ0JSTVBHWEowSDVqeUMxUHJwY3BTbGQ4RXk1eGJ6QU1CZ05WCkhSTUVCVEFEQVFIL01BMEdDU3FHU0liM0RRRUJDd1VBQTRJQkFRQTZoNXU4NnU1bVVnaVVMdGJrcE8vd0VUR0cKc1BaSzkxb0NZYkY2ajdNTHZZMGRBVVBOQmhCdHV5SlorTXRmL0c4YXVVSWVscjhabVBZcnpwc3BaTTBzcW5jRwo0MTF0Z1k4cDRVMHlWZnluY2RvVHVTNXJyRGlSemsxTG1rRnU5TVI4QmtYd3dab2hHUHdqdjhIVE12eUduN1QvCmYyRFRKWVJrMjhWZnEzOUZvZDhFaUxySDJPai9zeXE4RHJXVStLNzQwUFBSdjNnVkhiY2NJTVk1SkVnUVhGYk8KdTM3NUZ4TkkzaUthdFBWVktXaGNnQ1ZrVFQyTkZLbGduNU1MSTkrUDA4Zk5aZ1BEL1RPVUhTTlp0ZU0zUVdsRgpJTjM2YzhFOVFGZVJZQXhLRzlpNUkyODdKYk55RmEyTzFqT3IyNXhJT0FwcytBTzNVdmxNS1luVFFvUnMKLS0tLS1FTkQgQ0VSVElGSUNBVEUtLS0tLQ==
  tls.key: LS0tLS1CRUdJTiBQUklWQVRFIEtFWS0tLS0tCk1JSUV2UUlCQURBTkJna3Foa2lHOXcwQkFRRUZBQVNDQktjd2dnU2pBZ0VBQW9JQkFRQzEwRnJ4YXVhbzYzVDgKQnhNVE9MbXdmTnZDU2w4cThQNVlWM3RqSE1YekQvdDRhdi8wN2lSN2VKNGdNUDdWd3UwdWJwYXFuRHR2MXp5VQpOQlJPWHpjN253L0ZzLzhPUVpxT2dTU1dTTlREdU1zMkdlVFlYWVhoaGFzTHRIS2FmdHpGR29xK2pic0I3ZFdMCkxURkMydlFmd3VZTGZJVDVDNDNvKzcvck1IZDVHTEZIYkYyOS9MS2NsdnZpMEJvMFZiZ2hBR3RpcTdpL0FLZ2IKU3lQWktIMTZRU1c0S3hWZ2tCK0ZkM2ZmcjlJTGdtUkdrbG5aaHpUTkJBWFNvN3YzYTVvU2NjaysyeFEzQ0dmSAphSFIrMzk0OERVVFNNWndhdFVqV1NYd1FrNEhIbnJDMmxHS2FGcXA1YTdCMzNRbk5BMlcxTDBlSzRJblFCZ3p1CjVqclJMWkZKQWdNQkFBRUNnZ0VCQUlBT0d5KzBlbmQ1TjNFWFRvRVlUL2tWUi9jbFNMaEcrdTJmMkNEcnNDUk4KR25JcTh0SkJOT0xlQm9sbjJPVVlJTk8yUWxveUpjN1ZDeTdiSzM0enBBUlRPTm1udURvRFRPZVgzL3dWbWYycwpWenhvcW4wV0NlallxNnRYMGV3czlEUERTUHVEZWJhWEcvT2x4ZTl2cGNnRkR1QUNPT25zRUd5TGZCWWJrZHB3CnNOMGlWc2V3czdCSVZ0cVNZQ3hxWnQ0SG5YVXN0S3h2cDhoNDlmbEk0Q2ZHeXF1TmQzK3RPclQzMDJiM2JrMmYKbGdGUFBQZWpOa3hLemd3eThtRzRRS3V6L0FveVpvY2xMWG1tZTlqNkVnR0hFNjhwTTEwVDJoclNOVnd4dlJPdgpUUnVBdkVTS1d1QXNYcmEvMHFUOGpPTXFibmtpem5xL0dPcVF3eTIxSlRFQ2dZRUEzQWFUOVFSQjI1Q01VdlZiCkpXOTZEaUppZGg1NjJYY3VRVXk0Q0hvSjNGWEgyc3BvU25EbFdMMDJyL0haVEQ3S1pGb2ZBdE0xN3ZXYmxOVkQKNEdKMVB6ZUc2RllKc2tubDFaRkF4bVFLNUlMaDU5MkdjYjkvMUVaNDdHUlZ1THJUQjAxRHEvamJMZDJXN0FBMApLVkkyOUtkc0hNVlFTRzluNktlMkZaYmpWMlVDZ1lFQTA0cGhGNGFTQjdWSTMrcklTdDN4OS9zUW05cXFWTnNnCjdVYVRiUlh0NWtKZUJwMjk1VEM5aVdGL25DN1lHS0x6dTVHMTEwK2M4Qm92OXM0RVMzbXBRZE42QXlxWGNjcGYKd2RwaEZaL3hiVUxFTUZZaXIzY21Dcll6NDlJZFlOZkxvVEFNTkZBVVEwYTdNSkNVcElpVVZxZjVWOFpXb09ldQoxNzByd1I1eWJoVUNnWUJlQkdHTXZQajNQVmo0eWViY3BEbmZlRW5pZHp1L1dQSG03VVZtT2Qra3l2TFBSdFl4Cm1QUW5heExITSt6S2E3cElEMHBaQnE5eElIb09xdUtLS3dnMnJ4OXlmL0U4d2Q3MzEwMTZlSjArNzc1Z09ZU04KYm1yTTVpQWR3dkh1TWNhaXhCMDZpdWY5UFVhVUtiSWdSZ1U5blp6UTNtOWp4RmIrVDB1N0JmSzNFUUtCZ0JGawpNdzlJRlFGbE9oU09yT0RhMW1YWTBraTk5WHI4ZVB5STdOM3ZDNXYrb05SU2E2WklGNDBQNHZub01BUDJnYXRPCkUzRWNMbjdlNkVxV0xXdzNHRjg5RDY3cTlZQjVidDMxSWo0dzVCeGpmTldzZnpXRnlpN2ZRK3phL1dkVTFuenUKQWxMYXNvL0ZaT1pkUEpzNlQxZkFmNU5qendOemlNQmI0UmtEYzh6eEFvR0FML0tWVjZidHNCaFdSYk84Z2NadwprbnkwbE4xOXEwNUlQV2VYNjY2SS95RFFhV3BOcDlzNmZ4UGlDdXBGdFpDOUszY2xGVUFrbnZ3QmJPYUlCRnYzCjJTY05zaVBhaFUwMXJOSlNQR0EweVpJM3FyY0d3M1hVKzZ4SnFQYXdSMHhwd1E1a2tVS3ZLWGkvTGtrUjlrc0gKc2IzUkRjejZES0hvSTQ5QzBsaStyTDQ9Ci0tLS0tRU5EIFBSSVZBVEUgS0VZLS0tLS0=

创建并查看

kubectl apply -f *.yaml
kubectl get secrets 

创建ingress资源

apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  name: didi.cn-tls
spec:
  tls:
  - hosts:
    - nginx.didi.cn
    secretName: didi.cn
  rules:
    - host: nginx.didi.cn
      http:
        paths:
        - path: /
          backend:
            serviceName: web01
            servicePort: 443

创建并查看

kubectl apply -f *.yaml
kubectl get Ingress

浏览器访问