k8s部署express web应用（最新验证，手把手教学）
k8s部署node express web应用
本文档用于梳理k8s部署node应用的过程
-
准备项目
项目名称 开放端口 路由1 路由2 websvr1 3000 /web1/index /web1/send websvr2 3001 /web2/index /web2/send 为了快速搭建，此处的websvr采用express的脚手架express-generator安装：
#安装express-generator： $ npm install express-generator -g #创建脚手架应用app $ express app #安装依赖 $ cd app && npm install在app同级目录编辑Dockerfile:
#指定node版本 FROM node:10.15.1 #指定作者 MAINTAINER [SCH] #将同级app文件夹添加进入docker下指定目录 ADD app /opt/app #指定工作目录 WORKDIR /opt/app #指定对外端口 EXPOSE 3000 #启动执行命令 CMD [ "nohup","npm","start", "&"]将app与同级Dockerfile一起创建文件夹，命名为websvr1，修改app/routes/index.js路由文件：
//此处定义了两个get接口与一个post接口， router.get('/web1/index', function (req, res, next) { res.render('index', { title: 'Express1'}); }); router.post('/web1/getIndex', function (req, res, next) { res.send("get index1"); }); //通过websvr1向websvr2，service发起请求，用来验证k8s，service之间的通信流程 router.get('/web1/send', function (req, res, next) { request({ url: `http://websvr2-service:3001/web2/getIndex`, method: "POST", timeout: 10000 }, (error, response, body) => { if (error) { console.log(error); res.render('index', { title: "请求失败1"}); return } res.render('index', { title: body}); }) });另外拷贝一份websvr，将app目录下/bin/www内的默认端口和Dockerfile对外端口统一修改为3001，压缩命名为websvr2：
//此处定义了两个get接口与一个post接口， router.get('/web2/index', function (req, res, next) { res.render('index', { title: 'Express2'}); }); router.post('/web2/getIndex', function (req, res, next) { res.send("get index2"); }); //通过websvr1向websvr2，service发起请求，用来验证k8s，service之间的通信流程 router.get('/web2/send', function (req, res, next) { request({ url: `http://websvr1-service:3000/web1/getIndex`, method: "POST", timeout: 10000 }, (error, response, body) => { if (error) { console.log(error); res.render('index', { title: "请求失败2"}); return } res.render('index', { title: body}); }) });k8s集群条件
节点名称 IP k8s-master 172.16.66.169 k8s-node1 172.16.66.168 k8s-node2 172.16.66.170 -
创建docker镜像
将websvr1，websvr2分别上传至node1,node2下的/opt目录下，并创建docker镜像
$ cd /opt/websvr1 $ docker build -t websvr:v1 . $ cd /opt/websvr2 $ docker build -t websvr:v2 . #查看docker镜像 $ docker images REPOSITORY TAG IMAGE ID CREATED SIZE websvr v2 2a61bbea0d63 16 seconds ago 907MB websvr v1 a3adb933da80 32 seconds ago 907MB calico/node v3.20.1 355c1ee44040 4 weeks ago 156MB calico/pod2daemon-flexvol v3.20.1 55fa5eb71e09 4 weeks ago 21.7MB calico/cni v3.20.1 e69ccb66d1b6 4 weeks ago 146MB registry.aliyuncs.com/google_containers/kube-apiserver v1.21.0 4d217480042e 6 months ago 126MB registry.aliyuncs.com/google_containers/kube-proxy v1.21.0 38ddd85fe90e 6 months ago 122MB registry.aliyuncs.com/google_containers/kube-scheduler v1.21.0 62ad3129eca8 6 months ago 50.6MB registry.aliyuncs.com/google_containers/kube-controller-manager v1.21.0 09708983cc37 6 months ago 120MB registry.aliyuncs.com/google_containers/pause 3.4.1 0f8457a4c2ec 9 months ago 683kB coredns/coredns 1.8.0 296a6d5035e2 12 months ago 42.5MB registry.aliyuncs.com/google_containers/coredns/coredns v1.8.0 296a6d5035e2 12 months ago 42.5MB registry.aliyuncs.com/google_containers/etcd 3.4.13-0 0369cf4303ff 13 months ago 253MB node 10.15.1 8fc2110c6978 2 years ago 897MB -
k8s部署websvr
这里使用k8s，deployment，service部署websvr
deployment：创建docker容器群，对于同一个websvr，可以创建多个相同副本，通过分配不同虚拟IP及端口进行访问
service：当存在多个websvr容器副本后，如何通过统一的入口对多个websvr进行访问，就需要使用到service，可以简单理解为对多个容器副本的封装
$ vim websvr1.yamlapiVersion: apps/v1 kind: Deployment metadata: name: websvr1-deployment spec: selector: matchLabels: app: websvr1 replicas: 3 template: metadata: labels: app: websvr1 spec: containers: - name: websvr1 image: websvr:v1 ports: - containerPort: 3000 --- apiVersion: v1 kind: Service metadata: name: websvr1-service spec: selector: app: websvr1 ports: - protocol: TCP port: 3000 targetPort: 3000$ kubectl apply -f websvr1.yaml $ kubectl get pods -o wide NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES websvr1-deployment-7cb5776d76-mzx96 1/1 Running 0 3m8s 10.244.169.134 k8s-node2 <none> <none> websvr1-deployment-7cb5776d76-nzx7w 1/1 Running 0 3m8s 10.244.36.68 k8s-node1 <none> <none> websvr1-deployment-7cb5776d76-zzhdb 1/1 Running 0 3m8s 10.244.169.135 k8s-node2 <none> <none>同样方法部署websvr2，将暴露端口改为3001：
$ vim websvr2.yamlapiVersion: apps/v1 kind: Deployment metadata: name: websvr2-deployment spec: selector: matchLabels: app: websvr2 replicas: 3 template: metadata: labels: app: websvr2 spec: containers: - name: websvr2 image: websvr:v2 ports: - containerPort: 3001 --- apiVersion: v1 kind: Service metadata: name: websvr2-service spec: selector: app: websvr2 ports: - protocol: TCP port: 3001 targetPort: 3001$ kubectl apply -f websvr2.yaml $ kubectl get pods -o wide NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES websvr1-deployment-7cb5776d76-mzx96 1/1 Running 0 7m35s 10.244.169.134 k8s-node2 <none> <none> websvr1-deployment-7cb5776d76-nzx7w 1/1 Running 0 7m35s 10.244.36.68 k8s-node1 <none> <none> websvr1-deployment-7cb5776d76-zzhdb 1/1 Running 0 7m35s 10.244.169.135 k8s-node2 <none> <none> websvr2-deployment-58c8b7ffcd-57tsz 1/1 Running 0 7s 10.244.36.69 k8s-node1 <none> <none> websvr2-deployment-58c8b7ffcd-9lg4c 1/1 Running 0 7s 10.244.36.70 k8s-node1 <none> <none> websvr2-deployment-58c8b7ffcd-dgzl5 1/1 Running 0 7s 10.244.36.71 k8s-node1 <none> <none> -
验证
对于运行在各个node节点上的pod，通过统一的serviceIP及端口进行访问，service通过一定的负载均衡规则，分发到不同的node节点的pod上进行业务处理
$ kubectl get svc -o wide NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 135m <none> websvr1-service ClusterIP 10.102.171.58 <none> 3000/TCP 10m app=websvr1 websvr2-service ClusterIP 10.104.188.128 <none> 3001/TCP 2m34s app=websvr2 #发现此处有刚刚创建的两个websvr service，分别对应websvr1:3000及websvr2:3001 -
此时外网还无法访问k8s集群内容器，接下来需要进一步部署ingress
ingress-nginx部署
Ingress-nginx version k8s supported version Alpine Version Nginx Version v0.48.1 1.21, 1.20, 1.19 3.13.5 1.20.1 v0.47.0 1.21, 1.20, 1.19 3.13.5 1.20.1 v0.46.0 1.21, 1.20, 1.19 3.13.2 1.19.6 在master及所有node执行：
# 从阿里云镜像仓库拉取ingress-nginx所需版本： $ docker pull registry.cn-hangzhou.aliyuncs.com/kubernetes-fan/ingress-nginx:v0.48.1 # 将阿里云镜像重新打tag命名为官方镜像名： $ docker tag registry.cn-hangzhou.aliyuncs.com/kubernetes-fan/ingress-nginx:v0.48.1 k8s.gcr.io/ingress-nginx/controller:v0.48.1 # 删除阿里云镜像： $ docker rmi registry.cn-hangzhou.aliyuncs.com/kubernetes-fan/ingress-nginx:v0.48.1打开ingress-nginx 0.48.1的deploy.yaml网站、将yaml内容全部复制到本地。
https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v0.48.1/deploy/static/provider/baremetal/deploy.yaml
修改本地的deploy.yaml文件：
image: k8s.gcr.io/ingress-nginx/controller:v0.48.1@sha256:e9fb216ace49dfa4a5983b183067e97496e7a8b307d2093f4278cd550c303899 # 修改为 image: k8s.gcr.io/ingress-nginx/controller:v0.48.1外网无法打开可以使用下面保存的yaml文件：
apiVersion: v1 kind: Namespace metadata: name: ingress-nginx labels: app.kubernetes.io/name: ingress-nginx app.kubernetes.io/instance: ingress-nginx --- # Source: ingress-nginx/templates/controller-serviceaccount.yaml apiVersion: v1 kind: ServiceAccount metadata: labels: helm.sh/chart: ingress-nginx-3.34.0 app.kubernetes.io/name: ingress-nginx app.kubernetes.io/instance: ingress-nginx app.kubernetes.io/version: 0.48.1 app.kubernetes.io/managed-by: Helm app.kubernetes.io/component: controller name: ingress-nginx namespace: ingress-nginx automountServiceAccountToken: true --- # Source: ingress-nginx/templates/controller-configmap.yaml apiVersion: v1 kind: ConfigMap metadata: labels: helm.sh/chart: ingress-nginx-3.34.0 app.kubernetes.io/name: ingress-nginx app.kubernetes.io/instance: ingress-nginx app.kubernetes.io/version: 0.48.1 app.kubernetes.io/managed-by: Helm app.kubernetes.io/component: controller name: ingress-nginx-controller namespace: ingress-nginx data: --- # Source: ingress-nginx/templates/clusterrole.yaml apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRole metadata: labels: helm.sh/chart: ingress-nginx-3.34.0 app.kubernetes.io/name: ingress-nginx app.kubernetes.io/instance: ingress-nginx app.kubernetes.io/version: 0.48.1 app.kubernetes.io/managed-by: Helm name: ingress-nginx rules: - apiGroups: - '' resources: - configmaps - endpoints - nodes - pods - secrets verbs: - list - watch - apiGroups: - '' resources: - nodes verbs: - get - apiGroups: - '' resources: - services verbs: - get - list - watch - apiGroups: - extensions - networking.k8s.io # k8s 1.14+ resources: - ingresses verbs: - get - list - watch - apiGroups: - '' resources: - events verbs: - create - patch - apiGroups: - extensions - networking.k8s.io # k8s 1.14+ resources: - ingresses/status verbs: - update - apiGroups: - networking.k8s.io # k8s 1.14+ resources: - ingressclasses verbs: - get - list - watch --- # Source: ingress-nginx/templates/clusterrolebinding.yaml apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRoleBinding metadata: labels: helm.sh/chart: ingress-nginx-3.34.0 app.kubernetes.io/name: ingress-nginx app.kubernetes.io/instance: ingress-nginx app.kubernetes.io/version: 0.48.1 app.kubernetes.io/managed-by: Helm name: ingress-nginx roleRef: apiGroup: rbac.authorization.k8s.io kind: ClusterRole name: ingress-nginx subjects: - kind: ServiceAccount name: ingress-nginx namespace: ingress-nginx --- # Source: ingress-nginx/templates/controller-role.yaml apiVersion: rbac.authorization.k8s.io/v1 kind: Role metadata: labels: helm.sh/chart: ingress-nginx-3.34.0 app.kubernetes.io/name: ingress-nginx app.kubernetes.io/instance: ingress-nginx app.kubernetes.io/version: 0.48.1 app.kubernetes.io/managed-by: Helm app.kubernetes.io/component: controller name: ingress-nginx namespace: ingress-nginx rules: - apiGroups: - '' resources: - namespaces verbs: - get - apiGroups: - '' resources: - configmaps - pods - secrets - endpoints verbs: - get - list - watch - apiGroups: - '' resources: - services verbs: - get - list - watch - apiGroups: - extensions - networking.k8s.io # k8s 1.14+ resources: - ingresses verbs: - get - list - watch - apiGroups: - extensions - networking.k8s.io # k8s 1.14+ resources: - ingresses/status verbs: - update - apiGroups: - networking.k8s.io # k8s 1.14+ resources: - ingressclasses verbs: - get - list - watch - apiGroups: - '' resources: - configmaps resourceNames: - ingress-controller-leader-nginx verbs: - get - update - apiGroups: - '' resources: - configmaps verbs: - create - apiGroups: - '' resources: - events verbs: - create - patch --- # Source: ingress-nginx/templates/controller-rolebinding.yaml apiVersion: rbac.authorization.k8s.io/v1 kind: RoleBinding metadata: labels: helm.sh/chart: ingress-nginx-3.34.0 app.kubernetes.io/name: ingress-nginx app.kubernetes.io/instance: ingress-nginx app.kubernetes.io/version: 0.48.1 app.kubernetes.io/managed-by: Helm app.kubernetes.io/component: controller name: ingress-nginx namespace: ingress-nginx roleRef: apiGroup: rbac.authorization.k8s.io kind: Role name: ingress-nginx subjects: - kind: ServiceAccount name: ingress-nginx namespace: ingress-nginx --- # Source: ingress-nginx/templates/controller-service-webhook.yaml apiVersion: v1 kind: Service metadata: labels: helm.sh/chart: ingress-nginx-3.34.0 app.kubernetes.io/name: ingress-nginx app.kubernetes.io/instance: ingress-nginx app.kubernetes.io/version: 0.48.1 app.kubernetes.io/managed-by: Helm app.kubernetes.io/component: controller name: ingress-nginx-controller-admission namespace: ingress-nginx spec: type: ClusterIP ports: - name: https-webhook port: 443 targetPort: webhook selector: app.kubernetes.io/name: ingress-nginx app.kubernetes.io/instance: ingress-nginx app.kubernetes.io/component: controller --- # Source: ingress-nginx/templates/controller-service.yaml apiVersion: v1 kind: Service metadata: annotations: labels: helm.sh/chart: ingress-nginx-3.34.0 app.kubernetes.io/name: ingress-nginx app.kubernetes.io/instance: ingress-nginx app.kubernetes.io/version: 0.48.1 app.kubernetes.io/managed-by: Helm app.kubernetes.io/component: controller name: ingress-nginx-controller namespace: ingress-nginx spec: type: NodePort ports: - name: http port: 80 protocol: TCP targetPort: http - name: https port: 443 protocol: TCP targetPort: https selector: app.kubernetes.io/name: ingress-nginx app.kubernetes.io/instance: ingress-nginx app.kubernetes.io/component: controller --- # Source: ingress-nginx/templates/controller-deployment.yaml apiVersion: apps/v1 #kind: Deployment #apiVersion: extensions/v1beta1 # 修改为DaemonSet类型，随每个node节点创建和删除，配合污点容忍可以实现ingress-nginx高可用 kind: DaemonSet metadata: labels: helm.sh/chart: ingress-nginx-3.34.0 app.kubernetes.io/name: ingress-nginx app.kubernetes.io/instance: ingress-nginx app.kubernetes.io/version: 0.48.1 app.kubernetes.io/managed-by: Helm app.kubernetes.io/component: controller name: ingress-nginx-controller namespace: ingress-nginx spec: selector: matchLabels: app.kubernetes.io/name: ingress-nginx app.kubernetes.io/instance: ingress-nginx app.kubernetes.io/component: controller revisionHistoryLimit: 10 minReadySeconds: 0 template: metadata: labels: app.kubernetes.io/name: ingress-nginx app.kubernetes.io/instance: ingress-nginx app.kubernetes.io/component: controller spec: dnsPolicy: ClusterFirst #开启本机网络 hostNetwork: true containers: - name: controller image: k8s.gcr.io/ingress-nginx/controller:v0.48.1 imagePullPolicy: IfNotPresent lifecycle: preStop: exec: command: - /wait-shutdown args: - /nginx-ingress-controller - --election-id=ingress-controller-leader - --ingress-class=nginx - --configmap=$(POD_NAMESPACE)/ingress-nginx-controller - --validating-webhook=:8443 - --validating-webhook-certificate=/usr/local/certificates/cert - --validating-webhook-key=/usr/local/certificates/key #若本机端口被占用，需要另行设置 #- --http-port=81 #- --https-port=1444 #- --status-port=18081 securityContext: capabilities: drop: - ALL add: - NET_BIND_SERVICE runAsUser: 101 allowPrivilegeEscalation: true env: - name: POD_NAME valueFrom: fieldRef: fieldPath: metadata.name - name: POD_NAMESPACE valueFrom: fieldRef: fieldPath: metadata.namespace - name: LD_PRELOAD value: /usr/local/lib/libmimalloc.so livenessProbe: failureThreshold: 5 httpGet: path: /healthz port: 10254 scheme: HTTP initialDelaySeconds: 10 periodSeconds: 10 successThreshold: 1 timeoutSeconds: 1 readinessProbe: failureThreshold: 3 httpGet: path: /healthz port: 10254 scheme: HTTP initialDelaySeconds: 10 periodSeconds: 10 successThreshold: 1 timeoutSeconds: 1 ports: - name: http containerPort: 80 protocol: TCP - name: https containerPort: 443 protocol: TCP - name: webhook containerPort: 8443 protocol: TCP volumeMounts: - name: webhook-cert mountPath: /usr/local/certificates/ readOnly: true resources: requests: cpu: 100m memory: 90Mi nodeSelector: kubernetes.io/os: linux serviceAccountName: ingress-nginx terminationGracePeriodSeconds: 300 volumes: - name: webhook-cert secret: secretName: ingress-nginx-admission --- # Source: ingress-nginx/templates/admission-webhooks/validating-webhook.yaml # before changing this value, check the required kubernetes version # https://kubernetes.io/docs/reference/access-authn-authz/extensible-admission-controllers/#prerequisites apiVersion: admissionregistration.k8s.io/v1 kind: ValidatingWebhookConfiguration metadata: labels: helm.sh/chart: ingress-nginx-3.34.0 app.kubernetes.io/name: ingress-nginx app.kubernetes.io/instance: ingress-nginx app.kubernetes.io/version: 0.48.1 app.kubernetes.io/managed-by: Helm app.kubernetes.io/component: admission-webhook name: ingress-nginx-admission webhooks: - name: validate.nginx.ingress.kubernetes.io matchPolicy: Equivalent rules: - apiGroups: - networking.k8s.io apiVersions: - v1beta1 operations: - CREATE - UPDATE resources: - ingresses failurePolicy: Fail sideEffects: None admissionReviewVersions: - v1 - v1beta1 clientConfig: service: namespace: ingress-nginx name: ingress-nginx-controller-admission path: /networking/v1beta1/ingresses --- # Source: ingress-nginx/templates/admission-webhooks/job-patch/serviceaccount.yaml apiVersion: v1 kind: ServiceAccount metadata: name: ingress-nginx-admission namespace: ingress-nginx annotations: helm.sh/hook: pre-install,pre-upgrade,post-install,post-upgrade helm.sh/hook-delete-policy: before-hook-creation,hook-succeeded labels: helm.sh/chart: ingress-nginx-3.34.0 app.kubernetes.io/name: ingress-nginx app.kubernetes.io/instance: ingress-nginx app.kubernetes.io/version: 0.48.1 app.kubernetes.io/managed-by: Helm app.kubernetes.io/component: admission-webhook --- # Source: ingress-nginx/templates/admission-webhooks/job-patch/clusterrole.yaml apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRole metadata: name: ingress-nginx-admission annotations: helm.sh/hook: pre-install,pre-upgrade,post-install,post-upgrade helm.sh/hook-delete-policy: before-hook-creation,hook-succeeded labels: helm.sh/chart: ingress-nginx-3.34.0 app.kubernetes.io/name: ingress-nginx app.kubernetes.io/instance: ingress-nginx app.kubernetes.io/version: 0.48.1 app.kubernetes.io/managed-by: Helm app.kubernetes.io/component: admission-webhook rules: - apiGroups: - admissionregistration.k8s.io resources: - validatingwebhookconfigurations verbs: - get - update --- # Source: ingress-nginx/templates/admission-webhooks/job-patch/clusterrolebinding.yaml apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRoleBinding metadata: name: ingress-nginx-admission annotations: helm.sh/hook: pre-install,pre-upgrade,post-install,post-upgrade helm.sh/hook-delete-policy: before-hook-creation,hook-succeeded labels: helm.sh/chart: ingress-nginx-3.34.0 app.kubernetes.io/name: ingress-nginx app.kubernetes.io/instance: ingress-nginx app.kubernetes.io/version: 0.48.1 app.kubernetes.io/managed-by: Helm app.kubernetes.io/component: admission-webhook roleRef: apiGroup: rbac.authorization.k8s.io kind: ClusterRole name: ingress-nginx-admission subjects: - kind: ServiceAccount name: ingress-nginx-admission namespace: ingress-nginx --- # Source: ingress-nginx/templates/admission-webhooks/job-patch/role.yaml apiVersion: rbac.authorization.k8s.io/v1 kind: Role metadata: name: ingress-nginx-admission namespace: ingress-nginx annotations: helm.sh/hook: pre-install,pre-upgrade,post-install,post-upgrade helm.sh/hook-delete-policy: before-hook-creation,hook-succeeded labels: helm.sh/chart: ingress-nginx-3.34.0 app.kubernetes.io/name: ingress-nginx app.kubernetes.io/instance: ingress-nginx app.kubernetes.io/version: 0.48.1 app.kubernetes.io/managed-by: Helm app.kubernetes.io/component: admission-webhook rules: - apiGroups: - '' resources: - secrets verbs: - get - create --- # Source: ingress-nginx/templates/admission-webhooks/job-patch/rolebinding.yaml apiVersion: rbac.authorization.k8s.io/v1 kind: RoleBinding metadata: name: ingress-nginx-admission namespace: ingress-nginx annotations: helm.sh/hook: pre-install,pre-upgrade,post-install,post-upgrade helm.sh/hook-delete-policy: before-hook-creation,hook-succeeded labels: helm.sh/chart: ingress-nginx-3.34.0 app.kubernetes.io/name: ingress-nginx app.kubernetes.io/instance: ingress-nginx app.kubernetes.io/version: 0.48.1 app.kubernetes.io/managed-by: Helm app.kubernetes.io/component: admission-webhook roleRef: apiGroup: rbac.authorization.k8s.io kind: Role name: ingress-nginx-admission subjects: - kind: ServiceAccount name: ingress-nginx-admission namespace: ingress-nginx --- # Source: ingress-nginx/templates/admission-webhooks/job-patch/job-createSecret.yaml apiVersion: batch/v1 kind: Job metadata: name: ingress-nginx-admission-create namespace: ingress-nginx annotations: helm.sh/hook: pre-install,pre-upgrade helm.sh/hook-delete-policy: before-hook-creation,hook-succeeded labels: helm.sh/chart: ingress-nginx-3.34.0 app.kubernetes.io/name: ingress-nginx app.kubernetes.io/instance: ingress-nginx app.kubernetes.io/version: 0.48.1 app.kubernetes.io/managed-by: Helm app.kubernetes.io/component: admission-webhook spec: template: metadata: name: ingress-nginx-admission-create labels: helm.sh/chart: ingress-nginx-3.34.0 app.kubernetes.io/name: ingress-nginx app.kubernetes.io/instance: ingress-nginx app.kubernetes.io/version: 0.48.1 app.kubernetes.io/managed-by: Helm app.kubernetes.io/component: admission-webhook spec: containers: - name: create image: docker.io/jettech/kube-webhook-certgen:v1.5.1 imagePullPolicy: IfNotPresent args: - create - --host=ingress-nginx-controller-admission,ingress-nginx-controller-admission.$(POD_NAMESPACE).svc - --namespace=$(POD_NAMESPACE) - --secret-name=ingress-nginx-admission env: - name: POD_NAMESPACE valueFrom: fieldRef: fieldPath: metadata.namespace restartPolicy: OnFailure serviceAccountName: ingress-nginx-admission securityContext: runAsNonRoot: true runAsUser: 2000 --- # Source: ingress-nginx/templates/admission-webhooks/job-patch/job-patchWebhook.yaml apiVersion: batch/v1 kind: Job metadata: name: ingress-nginx-admission-patch namespace: ingress-nginx annotations: helm.sh/hook: post-install,post-upgrade helm.sh/hook-delete-policy: before-hook-creation,hook-succeeded labels: helm.sh/chart: ingress-nginx-3.34.0 app.kubernetes.io/name: ingress-nginx app.kubernetes.io/instance: ingress-nginx app.kubernetes.io/version: 0.48.1 app.kubernetes.io/managed-by: Helm app.kubernetes.io/component: admission-webhook spec: template: metadata: name: ingress-nginx-admission-patch labels: helm.sh/chart: ingress-nginx-3.34.0 app.kubernetes.io/name: ingress-nginx app.kubernetes.io/instance: ingress-nginx app.kubernetes.io/version: 0.48.1 app.kubernetes.io/managed-by: Helm app.kubernetes.io/component: admission-webhook spec: containers: - name: patch image: docker.io/jettech/kube-webhook-certgen:v1.5.1 imagePullPolicy: IfNotPresent args: - patch - --webhook-name=ingress-nginx-admission - --namespace=$(POD_NAMESPACE) - --patch-mutating=false - --secret-name=ingress-nginx-admission - --patch-failure-policy=Fail env: - name: POD_NAMESPACE valueFrom: fieldRef: fieldPath: metadata.namespace restartPolicy: OnFailure serviceAccountName: ingress-nginx-admission securityContext: runAsNonRoot: true runAsUser: 2000在master执行：
$ kubectl apply -f deploy.yaml $ kubectl get pod -o wide -n ingress-nginx NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES ingress-nginx-admission-create-87rgx 0/1 Completed 0 72s 10.244.169.137 k8s-node2 <none> <none> ingress-nginx-admission-patch-hq6b6 0/1 Completed 0 72s 10.244.36.74 k8s-node1 <none> <none> ingress-nginx-controller-f7d7r 1/1 Running 0 72s 172.16.66.170 k8s-node2 <none> <none> ingress-nginx-controller-p2z5t 1/1 Running 0 72s 172.16.66.168 k8s-node1 <none> <none> #可以看到ingress已经跟随node节点创建了两个controller用以监听nginx配置文件变化并更新 -
配置ingress
ingress-nginx安装完成后，还需要配置ingress路由规则，类似nginx的路由规则：
$ vim ingressRule.yamlapiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: my-ingress namespace: default annotations: kubernetes.io/ingress.class: "nginx" spec: rules: - host: k8s.test.com #指定域名 http: paths: - path: /web1 #一级路由 pathType: Prefix #匹配规则 Prefix：前缀 backend: service: name: websvr1-service #指向的service port: number: 3000 #对应的service暴露的端口 - path: /web2 pathType: Prefix backend: service: name: websvr2-service port: number: 3001$ kubectl apply -f ingressRule.yaml $ kubectl describe ingress Name: my-ingress Namespace: default Address: Default backend: default-http-backend:80 (<error: endpoints "default-http-backend" not found>) Rules: Host Path Backends ---- ---- -------- k8s.scbczx.com /web1 websvr1-service:3000 (10.244.169.134:3000,10.244.169.135:3000,10.244.36.68:3000) /web2 websvr2-service:3001 (10.244.169.136:3001,10.244.36.72:3001,10.244.36.73:3001) Annotations: kubernetes.io/ingress.class: nginx Events: Type Reason Age From Message ---- ------ ---- ---- ------- Normal Sync 11s nginx-ingress-controller Scheduled for sync Normal Sync 11s nginx-ingress-controller Scheduled for sync -
验证
此时通过curl发起get请求验证ingress-nginx路由规则
$ curl k8s.test.com/web1/index <!DOCTYPE html><html><head><title>Express1</title><link rel="stylesheet" href="/stylesheets/style.css"></head><body><h1>Express1</h1><p>Welcome to Express1</p></body></html> $ curl k8s.test.com/web1/send <!DOCTYPE html><html><head><title>get index2</title><link rel="stylesheet" href="/stylesheets/style.css"></head><body><h1>get index2</h1><p>Welcome to get index2</p></body></html> $ curl k8s.test.com/web2/index <!DOCTYPE html><html><head><title>Express2</title><link rel="stylesheet" href="/stylesheets/style.css"></head><body><h1>Express2</h1><p>Welcome to Express2</p></body></html> $ curl k8s.test.com/web2/send <!DOCTYPE html><html><head><title>get index1</title><link rel="stylesheet" href="/stylesheets/style.css"></head><body><h1>get index1</h1><p>Welcome to get index1</p></body></html>至此，k8s集群内的websvr都成功的通过公网域名进行访问
附：在实际的项目进展中，存在一些在当前线程内存中存储用户登录态的情况，比如服务器session，如果按照当前的service分发规则，很有可能导致用户登录session丢失的问题，那么service是否可以像nginx一样配置分发规则，比如按照前端IP？后面会在别的文档内单独讨论。
如有问题，欢迎指正。