K8S暴露pod内多个端口

 K8S暴露pod内多个端口

一、背景

公司统一用的某个底包跑jar服务，只暴露了8080端口 

二、需求

由于有些服务在启动jar服务后，会启动多个端口，除了8080端口，还有别的端口需要暴露，我这里就还需要暴露9999端口。

注：解决办法其实是可以直接改底包就好了，在底包中多暴露几个端口，但是我这边因为无法改底包，所以只能通过下面的办法解决。

三、解决办法

我们平时在打版升级的过程中，会基于底包写dockerfile来替换最新的jar包得到最终的镜像，所以可以这个dockerfile中添加暴露9999端口，这样同样也可以增加容器端口暴露，如下

$ cat dockerfile 
FROM 10.0.8.56/basis-images/basis:tomcat

ARG jar_name

RUN rm -rf /usr/local/tomcat/*
ADD ./target/${jar_name}.jar /usr/local/tomcat
ADD ./start.sh /usr/local/tomcat
EXPOSE 9999
RUN chmod +x /usr/local/tomcat/start.sh

$ docker build --build-arg jar_name=nsw-ai-video . -t 10.0.8.56/nsyai-test/nsw-ai-video:2023-07-28-15-40
$ docker push 10.0.8.56/nsyai-test/nsw-ai-video:2023-07-28-15-40

四、实验在docker上跑容器，验证是否暴露出8080和9999端口（10.0.8.56是我的harbor私有镜像仓库）

[ yukw @ docker-work01 10.0.8.59 ] ~
$ docker login 10.0.8.56
Username: yukw
Password: 
WARNING! Your password will be stored unencrypted in /home/yukw/.docker/config.json.
Configure a credential helper to remove this warning. See
https://docs.docker.com/engine/reference/commandline/login/#credentials-store

Login Succeeded
[ yukw @ docker-work01 10.0.8.59 ] ~
$ docker run -d -P --name nsyai-test 10.0.8.56/nsyai-test/nsw-ai-video:2023-07-28-15-40
Unable to find image '10.0.8.56/nsyai-test/nsw-ai-video:2023-07-28-15-40' locally
2023-07-28-15-40: Pulling from nsyai-test/nsw-ai-video
a8c7037c15e9: Pull complete 
7f59206c4cb3: Pull complete 
d6593d2ee432: Pull complete 
47613084598b: Pull complete 
7ef22be88035: Pull complete 
edf70be6f818: Pull complete 
af72e686cb89: Pull complete 
376658e1b07e: Pull complete 
6991c8295d7f: Pull complete 
f0a023d2bec5: Pull complete 
9356db0572c6: Pull complete 
1cbc500b22f4: Pull complete 
2a8383c1d611: Pull complete 
962207b93da3: Pull complete 
9fdef278ff07: Pull complete 
8cc25cf21f3b: Pull complete 
Digest: sha256:e07a648e671746f4408565b2237584303cfdfb7d5a451adfa707dda3fc87d670
Status: Downloaded newer image for 10.0.8.56/nsyai-test/nsw-ai-video:2023-07-28-15-40
e11553520d6c4d94c71d8d11a699bd4d1c6df8202d4e1ec15b28ca1bcd21ff25
[ yukw @ docker-work01 10.0.8.59 ] ~
$ docker ps -a |grep 'nsyai-test'
e11553520d6c   10.0.8.56/nsyai-test/nsw-ai-video:2023-07-28-15-40   "/usr/local/tomcat/s…"   8 seconds ago   Up 7 seconds   0.0.0.0:49154->8080/tcp, 0.0.0.0:49153->9999/tcp   nsyai-test
9fd678ee8eeb   10.0.8.56/nsyai-test/nsyai-web:2023-07-12-12-01      "/docker-entrypoint.…"   2 weeks ago     Up 2 weeks     80/tcp                                             my-nsyai-test
[ yukw @ docker-work01 10.0.8.59 ] ~
$ docker port e11553520d6c
8080/tcp -> 0.0.0.0:49154
9999/tcp -> 0.0.0.0:49153

实验发现，端口暴露成功
容器8080端口随机映射到了宿主机49154端口
容器9999端口随机映射到了宿主机49153端口

五、编写dp.yaml

# cat dp.yaml 
apiVersion: apps/v1
kind: Deployment
metadata:
  annotations:
    deployment.kubernetes.io/revision: "63"
    description: video模块
  labels:
    k8s-app: nsw-ai-video
    qcloud-app: nsw-ai-video
  name: nsw-ai-video
  namespace: nsyai-test
spec:
  progressDeadlineSeconds: 600
  replicas: 1
  revisionHistoryLimit: 10
  selector:
    matchLabels:
      k8s-app: nsw-ai-video
      qcloud-app: nsw-ai-video
  template:
    metadata:
      labels:
        k8s-app: nsw-ai-video
        qcloud-app: nsw-ai-video
    spec:
      containers:
      - name: nsw-ai-video
        image: 10.0.8.56/nsyai-test/nsw-ai-video:2023-07-28-15-40
        imagePullPolicy: Always
        livenessProbe:
          failureThreshold: 5
          initialDelaySeconds: 180
          periodSeconds: 10
          successThreshold: 1
          tcpSocket:
            port: 8080
          timeoutSeconds: 6
        readinessProbe:
          failureThreshold: 5
          initialDelaySeconds: 60
          periodSeconds: 10
          successThreshold: 1
          tcpSocket:
            port: 8080
          timeoutSeconds: 60
        resources:
          limits:
            cpu: 2000m
            memory: 2Gi
          requests:
            cpu: "1"
            memory: 512Mi
        ports:
        - containerPort: 8080
          name: image-port
          protocol: TCP
        - containerPort: 9999
          name: xxl-job-port
          protocol: TCP
      imagePullSecrets:
      - name: nsw-harbor-secret 

containerPort是在pod控制器中定义的、pod中的容器需要暴露的端口 

六、编写svc.yaml

# cat svc.yaml 
apiVersion: v1
kind: Service
metadata:
  name: nsw-ai-video
  namespace: nsyai-test
spec:
  externalTrafficPolicy: Cluster
  ports:
  - name: 8080-8080-tcp
    nodePort: 30083
    port: 8080
    protocol: TCP
    targetPort: 8080
  - name: 9999-9999-tcp
    nodePort: 30084
    port: 9999
    protocol: TCP
    targetPort: 9999
  selector:
    k8s-app: nsw-ai-video
    qcloud-app: nsw-ai-video
  type: NodePort

七、应用配置清单

# kubectl apply -f dp.yaml
# kubectl apply -f svc.yaml

# kubectl get svc -n nsyai-test
NAME             TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)                         AGE
nsw-ai-gateway   ClusterIP   10.0.0.107           8080/TCP                        16d
nsw-ai-video     NodePort    10.0.0.119           8080:30083/TCP,9999:30084/TCP   3h25m
nsyai-pc-nginx   NodePort    10.0.0.185           80:30082/TCP                    16d

总结：

1、从上面可以发现，在制作pod镜像中EXPOSE暴露了两个端口，这个是容器本身需要暴露的端口，在dp.yaml中配置了两个containerPort，这个是pod中的容器需要暴露的端口，在svc.yaml中配置了nodePort，port，targetport，分别代表宿主机端口，service端口和容器端口。

  好了，这就是K8S暴露pod内多个端口的办法了，如有问题可与博主一起交流讨论！