kubernetes ingress方式访问集群服务

目标：通过Ingress方式访问集群服务

环境：Kubenetes集群 / Linux

步骤：概述->部署Kubernetes Ingress容器->编写jupyter部署yaml文件->运行测试

1.概述

应用部署到Kubernetes集群之后，需要对外发布服务，目前Kubernetes支持三种方式暴露服务：

(1)NodePort

Service通过与集群节点的端口映射进行服务发布，外部用户需要获取节点IP，并以http://NodeIP:port的方式访问

(2)LoadBlancer

通过设置公网负载均衡器地址进行访问

(3)Ingress

Ingress方式能通过Nginx、HAproxy等负载均衡器暴露集群内服务，包括Ingress Controller与Ingress两部分。

Ingress Contronler 通过与 Kubernetes API 交互，动态的去感知集群中 Ingress 规则变化，然后读取它，按照自定义的规则，规则就是写明了哪个域名对应哪个service，生成一段 Nginx 配置，再写到 Nginx-ingress-control的 Pod 里，这个 Ingress Contronler 的pod里面运行着一个nginx服务，控制器会把生成的nginx配置写入/etc/nginx.conf文件中，然后 reload 一下 使用配置生效。以此来达到域名分配置及动态更新的问题。

本文使用以nginx作为负载均衡的Ingress

2.部署Kubernetes Ingress容器

由于测试k8s环境由kubeadm方式部署，因此读取的github官方配置文件地址为：

https://github.com/kubernetes/ingress-nginx/tree/a3131c569f6b0d6502dda1548f63be4b56e36e42/examples/deployment/nginx

选用kubeadm目录下的文件nginx-ingress-controller.yaml

(1)nginx-ingress-controller.yaml

原始文件需要修改，加入serviceaccount的配置，不让会导致controller无法访问集群API

nginx-ingress-controller.yaml

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: default-http-backend
  labels:
    k8s-app: default-http-backend
  namespace: kube-system
spec:
  replicas: 1
  template:
    metadata:
      labels:
        k8s-app: default-http-backend
    spec:
      terminationGracePeriodSeconds: 60
      containers:
      - name: default-http-backend
        # Any image is permissable as long as:
        # 1. It serves a 404 page at /
        # 2. It serves 200 on a /healthz endpoint
        image: gcr.io/google_containers/defaultbackend:1.0
        livenessProbe:
          httpGet:
            path: /healthz
            port: 8080
            scheme: HTTP
          initialDelaySeconds: 30
          timeoutSeconds: 5
        ports:
        - containerPort: 8080
        resources:
          limits:
            cpu: 10m
            memory: 20Mi
          requests:
            cpu: 10m
            memory: 20Mi
---
apiVersion: v1
kind: Service
metadata:
  name: default-http-backend
  namespace: kube-system
  labels:
    k8s-app: default-http-backend
spec:
  ports:
  - port: 80
    targetPort: 8080
  selector:
    k8s-app: default-http-backend
---
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: nginx-ingress-controller
  labels:
    k8s-app: nginx-ingress-controller
  namespace: kube-system
spec:
  replicas: 1
  template:
    metadata:
      labels:
        k8s-app: nginx-ingress-controller
    spec:
      # hostNetwork makes it possible to use ipv6 and to preserve the source IP correctly regardless of docker configuration
      # however, it is not a hard dependency of the nginx-ingress-controller itself and it may cause issues if port 10254 already is taken on the host
      # that said, since hostPort is broken on CNI (https://github.com/kubernetes/kubernetes/issues/31307) we have to use hostNetwork where CNI is used
      # like with kubeadm
      serviceAccountName: nginx-ingress-serviceaccount
      hostNetwork: true
      terminationGracePeriodSeconds: 60
      containers:
      - image: gcr.io/google_containers/nginx-ingress-controller:0.9.0-beta.11
        name: nginx-ingress-controller
        readinessProbe:
          httpGet:
            path: /healthz
            port: 10254
            scheme: HTTP
        livenessProbe:
          httpGet:
            path: /healthz
            port: 10254
            scheme: HTTP
          initialDelaySeconds: 10
          timeoutSeconds: 1
        ports:
        - containerPort: 80
          hostPort: 80
        - containerPort: 443
          hostPort: 443
        env:
          - name: POD_NAME
            valueFrom:
              fieldRef:
                fieldPath: metadata.name
          - name: POD_NAMESPACE
            valueFrom:
              fieldRef:
                fieldPath: metadata.namespace
        args:
        - /nginx-ingress-controller
        - --default-backend-service=$(POD_NAMESPACE)/default-http-backend

其中包含gcr镜像，因此需要拉取国内镜像重新打标签

ingress-controller-image.sh

docker pull anjia0532/google-containers.defaultbackend:1.0
docker pull anjia0532/google-containers.nginx-ingress-controller:0.9.0-beta.11
 
docker tag anjia0532/google-containers.defaultbackend:1.0 gcr.io/google_containers/defaultbackend:1.0
docker tag anjia0532/google-containers.nginx-ingress-controller:0.9.0-beta.11 gcr.io/google_containers/nginx-ingress-controller:0.9.0-beta.11

(2)rbac.yaml

该文件的作用为创建服务账户，赋予nginx-controller的kubernetes api访问权限

rbac.yaml

apiVersion: v1
kind: ServiceAccount
metadata:
  name: nginx-ingress-serviceaccount
  namespace: kube-system
 
---
 
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRole
metadata:
  name: nginx-ingress-clusterrole
rules:
  - apiGroups:
      - ""
    resources:
      - configmaps
      - endpoints
      - nodes
      - pods
      - secrets
    verbs:
      - list
      - watch
  - apiGroups:
      - ""
    resources:
      - nodes
    verbs:
      - get
  - apiGroups:
      - ""
    resources:
      - services
    verbs:
      - get
      - list
      - watch
  - apiGroups:
      - "extensions"
    resources:
      - ingresses
    verbs:
      - get
      - list
      - watch
  - apiGroups:
      - ""
    resources:
        - events
    verbs:
        - create
        - patch
  - apiGroups:
      - "extensions"
    resources:
      - ingresses/status
    verbs:
      - update
 
---
 
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: Role
metadata:
  name: nginx-ingress-role
  namespace: kube-system
rules:
  - apiGroups:
      - ""
    resources:
      - configmaps
      - pods
      - secrets
      - namespaces
    verbs:
      - get
  - apiGroups:
      - ""
    resources:
      - configmaps
    resourceNames:
      - "ingress-controller-leader-nginx"
    verbs:
      - get
      - update
  - apiGroups:
      - ""
    resources:
      - configmaps
    verbs:
      - create
  - apiGroups:
      - ""
    resources:
      - endpoints
    verbs:
      - get
      - create
 
---
 
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: RoleBinding
metadata:
  name: nginx-ingress-role-nisa-binding
  namespace: kube-system
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: nginx-ingress-role
subjects:
  - kind: ServiceAccount
    name: nginx-ingress-serviceaccount
    namespace: kube-system
 
---
 
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRoleBinding
metadata:
  name: nginx-ingress-clusterrole-nisa-binding
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: nginx-ingress-clusterrole
subjects:
  - kind: ServiceAccount
    name: nginx-ingress-serviceaccount
    namespace: kube-system

github原始文件地址：https://github.com/kubernetes/ingress-nginx/blob/master/deploy/rbac.yaml

将其中的命名空间修改为kube-system

(3)部署

执行kubectl create -f *.yaml命令

分别查看controller和default-backend信息

kubectl get pods  --namespace=kube-system -l k8s-app=default-http-backend -o wid

kubectl get pods  --namespace=kube-system -l k8s-app=nginx-ingress-controller -o wide

3.编写jupyter部署yaml文件

基于jupyter镜像编写yaml文件，提供deployment、service、ingress三种类型的资源

jupyter-deploy.yaml

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: jupyter-deployment
spec:
  replicas: 1
  template:
    metadata:
      labels:
        app: jupyter
    spec:
      containers:
      - name: jupyter-test
        image: jupyter/tensorflow-notebook
        ports:
        - containerPort: 8888
   
---
apiVersion: v1
kind: Service
metadata:
  name: jupyter-service
  labels:
    app: jupyter
spec:
  selector:
    app: jupyter
  ports:
  - port: 8888
    targetPort: 8888
 
---
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: jupyter-ingress
spec:
  rules:
  - host: test.jupyter.com
    http:
      paths:
      - backend:
          serviceName: jupyter-service
          servicePort: 8888

其中设置ingress后端的访问域名为test.jupyter.com

ingress部署信息查看：

kubectl get ing

4.运行测试

在客户端主机上加入对域名的解析，即test.jupyter.com node4's IP

访问test.jupyter.com

以上，完成测试。