k8s Ingress 服务部署方式

k8s Ingress 服务部署方式

通常情况下，service和pod仅可在集群内部网络中通过IP地址访问。所有到达边界路由器的流量或被丢弃或被转发到其他地方

官网对 Ingress 的定义为管理对外服务到集群内服务之间规则的集合，通俗点讲就是它定义规则来允许进入集群的请求被转发到集群中对应服务上，从来实现服务暴漏。Ingress 能把集群内 Service 配置成外网能够访问的 URL，流量负载均衡，终止SSL，提供基于域名访问的虚拟主机等等。

Ingress

Ingress 使用开源的反向代理负载均衡器来实现对外暴漏服务，比如 Nginx、Apache、Haproxy等。Nginx Ingress 一般有三个组件组成：

Nginx 反向代理负载均衡器

Ingress Controller

Ingress Controller 可以理解为控制器，它通过不断的跟 Kubernetes API 交互，实时获取后端 Service、Pod 等的变化，比如新增、删除等，然后结合 Ingress 定义的规则生成配置，然后动态更新上边的 Nginx 负载均衡器，并刷新使配置生效，来达到服务自动发现的作用。

Ingress

Ingress 则是定义规则，通过它定义某个域名的请求过来之后转发到集群中指定的 Service。它可以通过 Yaml 文件定义，可以给一个或多个 Service 定义一个或多个 Ingress 规则。

以上三者有机的协调配合起来，就可以完成 Kubernetes 集群服务的暴漏

Kubernetes 使用 Nginx Ingress 暴漏tomcat服务，前提我们需要有一个正常运行的集群服务，根据之前创建的的kubernetes集群，进行测试

官网地址：https://kubernetes.io/docs/concepts/services-networking/ingress/

　　　　https://github.com/kubernetes/ingress-nginx/

使用到的镜像

gcr.io/google_containers/defaultbackend:1.4

quay.io/kubernetes-ingress-controller/nginx-ingress-controller:0.18.0

ingress-nginx组件有几个部分组成：

configmap.yaml:提供configmap可以在线更行nginx的配置

default-backend.yaml:提供一个缺省的后台错误页面 404

namespace.yaml:创建一个独立的命名空间 ingress-nginx

rbac.yaml:创建对应的role rolebinding 用于rbac

tcp-services-configmap.yaml:修改L4负载均衡配置的configmap

udp-services-configmap.yaml:修改L4负载均衡配置的configmap

nginx-ingress-controller.yaml:有应用rbac的nginx-ingress-controller组件

YAML配置文件

namespace.yaml

apiVersion: v1

kind: Namespace

metadata:

  name: ingress-nginx

default-backend.yaml

apiVersion: extensions/v1beta1

kind: Deployment

metadata:

  name: default-http-backend

  labels:

   app.kubernetes.io/name: default-http-backend

   app.kubernetes.io/part-of: ingress-nginx

  namespace: ingress-nginx

spec:

  replicas: 1

  selector:

   matchLabels:

     app.kubernetes.io/name: default-http-backend

     app.kubernetes.io/part-of: ingress-nginx

  template:

   metadata:

     labels:

       app.kubernetes.io/name: default-http-backend

       app.kubernetes.io/part-of: ingress-nginx

   spec:

     terminationGracePeriodSeconds: 60

     containers:

       - name: default-http-backend

         image: googlecontainer/defaultbackend-amd64:1.4

         #image:

         livenessProbe:

           httpGet:

             path: /healthz

             port: 8080

             scheme: HTTP

           initialDelaySeconds: 30

           timeoutSeconds: 5

         ports:

           - containerPort: 8080

         resources:

           limits:

             cpu: 10m

             memory: 20Mi

           requests:

             cpu: 10m

             memory: 20Mi

​

---

apiVersion: v1

kind: Service

metadata:

  name: default-http-backend

  namespace: ingress-nginx

  labels:

   app.kubernetes.io/name: default-http-backend

   app.kubernetes.io/part-of: ingress-nginx

spec:

  ports:

   - port: 80

     targetPort: 8080

  selector:

   app.kubernetes.io/name: default-http-backend

   app.kubernetes.io/part-of: ingress-nginx

---

​

configmap.yaml

kind: ConfigMap

apiVersion: v1

metadata:

  name: nginx-configuration

  namespace: ingress-nginx

  labels:

   app: ingress-nginx

tcp-services-configmap.yaml

kind: ConfigMap

apiVersion: v1

metadata:

  name: tcp-services

  namespace: ingress-nginx

udp-services-configmap.yaml

kind: ConfigMap

apiVersion: v1

metadata:

  name: udp-services

  namespace: ingress-nginx

rbac.yaml

apiVersion: v1

kind: ServiceAccount

metadata:

  name: nginx-ingress-serviceaccount

  namespace: ingress-nginx

  labels:

   app.kubernetes.io/name: ingress-nginx

   app.kubernetes.io/part-of: ingress-nginx

​

---

apiVersion: rbac.authorization.k8s.io/v1beta1

kind: ClusterRole

metadata:

  name: nginx-ingress-clusterrole

  labels:

   app.kubernetes.io/name: ingress-nginx

   app.kubernetes.io/part-of: ingress-nginx

rules:

  - apiGroups:

     - ""

   resources:

     - configmaps

     - endpoints

     - nodes

     - pods

     - secrets

   verbs:

     - list

     - watch

  - apiGroups:

     - ""

   resources:

     - nodes

   verbs:

     - get

  - apiGroups:

     - ""

   resources:

     - services

   verbs:

     - get

     - list

     - watch

  - apiGroups:

     - "extensions"

   resources:

     - ingresses

   verbs:

     - get

     - list

     - watch

  - apiGroups:

     - ""

   resources:

     - events

   verbs:

     - create

     - patch

  - apiGroups:

     - "extensions"

   resources:

     - ingresses/status

   verbs:

     - update

​

---

apiVersion: rbac.authorization.k8s.io/v1beta1

kind: Role

metadata:

  name: nginx-ingress-role

  namespace: ingress-nginx

  labels:

   app.kubernetes.io/name: ingress-nginx

   app.kubernetes.io/part-of: ingress-nginx

rules:

  - apiGroups:

     - ""

   resources:

     - configmaps

     - pods

     - secrets

     - namespaces

   verbs:

     - get

  - apiGroups:

     - ""

   resources:

     - configmaps

   resourceNames:

     # Defaults to "-"

     # Here: "-"

     # This has to be adapted if you change either parameter

     # when launching the nginx-ingress-controller.

     - "ingress-controller-leader-nginx"

   verbs:

     - get

     - update

  - apiGroups:

     - ""

   resources:

     - configmaps

   verbs:

     - create

  - apiGroups:

     - ""

   resources:

     - endpoints

   verbs:

     - get

​

---

apiVersion: rbac.authorization.k8s.io/v1beta1

kind: RoleBinding

metadata:

  name: nginx-ingress-role-nisa-binding

  namespace: ingress-nginx

  labels:

   app.kubernetes.io/name: ingress-nginx

   app.kubernetes.io/part-of: ingress-nginx

roleRef:

  apiGroup: rbac.authorization.k8s.io

  kind: Role

  name: nginx-ingress-role

subjects:

  - kind: ServiceAccount

   name: nginx-ingress-serviceaccount

   namespace: ingress-nginx

​

---

apiVersion: rbac.authorization.k8s.io/v1beta1

kind: ClusterRoleBinding

metadata:

  name: nginx-ingress-clusterrole-nisa-binding

  labels:

   app.kubernetes.io/name: ingress-nginx

   app.kubernetes.io/part-of: ingress-nginx

roleRef:

  apiGroup: rbac.authorization.k8s.io

  kind: ClusterRole

  name: nginx-ingress-clusterrole

subjects:

  - kind: ServiceAccount

   name: nginx-ingress-serviceaccount

   namespace: ingress-nginx

​

---

nginx-ingress-controller.yaml

apiVersion: extensions/v1beta1

kind: DaemonSet

metadata:

  name: nginx-ingress-controller

  namespace: ingress-nginx

spec:

  selector:

   matchLabels:

     app: ingress-nginx

  template:

   metadata:

     labels:

       app: ingress-nginx

     annotations:

       prometheus.io/port: '10254'

       prometheus.io/scrape: 'true'

   spec:

     serviceAccountName: nginx-ingress-serviceaccount

     hostNetwork: true

     containers:

       - name: nginx-ingress-controller

         image: quay.io/kubernetes-ingress-controller/nginx-ingress-controller:0.18.0

         args:

           - /nginx-ingress-controller

           - --default-backend-service=$(POD_NAMESPACE)/default-http-backend

           - --configmap=$(POD_NAMESPACE)/nginx-configuration

           - --tcp-services-configmap=$(POD_NAMESPACE)/tcp-services

           - --udp-services-configmap=$(POD_NAMESPACE)/udp-services

           - --annotations-prefix=nginx.ingress.kubernetes.io

         env:

           - name: POD_NAME

             valueFrom:

               fieldRef:

                 fieldPath: metadata.name

           - name: POD_NAMESPACE

             valueFrom:

               fieldRef:

                 fieldPath: metadata.namespace

         ports:

         - name: http

           containerPort: 80

         - name: https

           containerPort: 443

         livenessProbe:

           failureThreshold: 3

           httpGet:

             path: /healthz

             port: 10254

             scheme: HTTP

           initialDelaySeconds: 10

           periodSeconds: 10

           successThreshold: 1

           timeoutSeconds: 1

         readinessProbe:

           failureThreshold: 3

           httpGet:

             path: /healthz

             port: 10254

             scheme: HTTP

           periodSeconds: 10

           successThreshold: 1

           timeoutSeconds: 1

     #nodeSelector:

       #custom/ingress-controller-ready: "true"

说明：

kind: DaemonSet：官方文档给的是deployment，replicate 为 1，这样将会在某一台节点上启动对应的nginx-ingress-controller pod。外部流量访问至该节点，由该节点负载分担至内部的service。测试环境考虑防止单点故障，改为DaemonSet，配合亲和性部署在制定节点上启动nginx-ingress-controller pod，确保有多个节点启动nginx-ingress-controller pod，后续将这些节点加入到外部硬件负载均衡组实现高可用性。

hostNetwork: true：添加该字段，暴露nginx-ingress-controller pod的服务端口（80）

nodeSelector: 加入亲和性部署，有custom/ingress-controller-ready label的才会部署该DaemonSet（未使用）

加载yaml文件

kubectl create -f .

查看pod创建情况

#kubectl get pod,svc,deployment -n ingress-nginx -o wide

在上面已经部署了 ingress-nginx，下面我们部署一个tomcat作为对外暴露的服务

使用ingress部署一个对外暴露的服务包含一个pod和一个service

创建tomcat service yaml

vi tomcat-deploy.yaml

复制下面的内容：

apiVersion: v1

kind: Service

metadata:

  name: tomcat

  namespace: default

spec:

  selector:

   app: tomcat

   release: canary

  ports:

  - name: http

   targetPort: 8080

   port: 8080

  - name: ajp

   targetPort: 8009

   port: 8009

​

---

apiVersion: apps/v1

kind: Deployment

metadata:

  name: tomcat-deploy

  namespace: default

spec:

  replicas: 1

  selector:

   matchLabels:

     app: tomcat

     release: canary

  template:

   metadata:

     labels:

       app: tomcat

       release: canary

   spec:

     containers:

     - name: tomcat

       image: tomcat

       ports:

       - name: http

         containerPort: 8080

创建tomcat pod yaml文件

vi ingress-tomcat.yaml

复制下面内容：

apiVersion: extensions/v1beta1

kind: Ingress

metadata:

  name: ingress-tomcat

  namespace: default

  annotations:

   kubernetes.io/ingress.class: "nginx"

spec:

  rules:

  - host: k8s.master ##这个地方使用主机名或者域名，不能使用IP地址

   http:

     paths:

     - path:

       backend:

         serviceName: tomcat

         servicePort: 8080

然后执行：

kubectl apply -f tomcat-deploy.yaml

​

kubectl apply -f ingress-tomcat.yaml

服务启动以后就可以通过上面配置的主机名或者域名进行访问tomcat 了

http://k8s.master/