K8S Ingress-Nginx导出TCP端口

ingress-nginx导出TCP端口

• Exposing TCP and UDP services - Ingress-Nginx Controller
• https://github.com/kubernetes/ingress-nginx/blob/main/docs/user-guide/exposing-tcp-udp-services.md

• helm upgrade ingress-nginx导出redis 6379端口（这种方式最简单，接下为我们研究一下主要的流程原理）

• • K8s and Redis; a tale of Layer 4 Ingress

helm upgrade ingress-nginx ingress-nginx/ingress-nginx -n ingress-nginx \
  --values values.yaml \
  --set tcp.26379="develop/redis-cluster:6379" --debug

• nginx-ingress 已经能够完成 7/4 层的代理功能（4 层代理基于 ConfigMap）
• 流程图，请参考：Kubernetes K8S 1.20部署Ingress nginx 0.30_k8s1.20.9部署ingress-nginx-CSDN博客

• 整体流程大概是这样子的

  • SVC(NodePort:30009映射到后台26379端口，26379端口转发到26379-tcp)
    • DaemonSet/ingress-nginx-controller(26379-tcp转到controller容器的端口containerPort: 26379)
      • ingress-nginx-controller容器中有nginx，nginx监听26379端口
        • 从configMap中取的26379端口要转发的地址
          • develop/redis-cluster:6379
  • 用户请求流程
    • 访问30009NodePort端口
      • 映射到26379端口
        • ingress-nginx-controller中的nginx监听26379端口
          • 将26379端口转发至develop/redis-cluster:6379

• 添加configMap

vi configMap.yaml

apiVersion: v1
kind: ConfigMap
metadata:
  name: tcp-services
  namespace: ingress-nginx
data:
  26379: "develop/redis-cluster:6379" #从26379的TCP端口转发到Redis集群的6379端口，至于nodePort是哪个还未定

kubectl apply -f configMap.yaml -n ingress-nginx

也可以自己配置SVC

编辑service, 即ClusterIP
  kubectl edit svc -n ingress-nginx ingress-nginx-controller
  - name: 26379-tcp #配置一个新的TCP端口，取名为26379-tcp
    nodePort: 30009 #导出对外的端口
    port: 26379 #监听26379端口
    protocol: TCP
    targetPort: 26379-tcp #转发到ds上的26379-tcp端口

配置DaemonSet

• ingress-nginx-controller pod实际是Controlled By: DaemonSet/ingress-nginx-controller
• 修改DaemonSet/ingress-nginx-controller
  • 将svc上监听到的26379端口转发到DaemonSet/ingress-nginx-controller上
• DaemonSet/ingress-nginx-controller连接到ingress-nginx-controller对应的pod容器的26379端口上

 kubectl edit ds -n ingress-nginx ingress-nginx-controller

 - --tcp-services-configmap=$(POD_NAMESPACE)/tcp-services

        ports:
        - containerPort: 80
          name: http
          protocol: TCP
        - containerPort: 443
          name: https
          protocol: TCP
        - containerPort: 8443
          name: webhook
          protocol: TCP
        - containerPort: 26379
          name: 26379-tcp
          protocol: TCP
          
          
#查看SVC，已经有了26379端口，并绑定了nodePort端口30009
[root@k8s-master01 ingress-nginx]# kubectl get svc -n ingress-nginx
NAME                                 TYPE           CLUSTER-IP      EXTERNAL-IP   PORT(S)                                     AGE
ingress-nginx-controller             LoadBalancer   10.99.163.44         80:31268/TCP,443:31052/TCP,26379:30009/TCP   3d11h
ingress-nginx-controller-admission   ClusterIP      10.100.131.12           443/TCP                                  

查看nginx.conf代理配置

• kubectl exec -it ingress-nginx-controller-lkzzh /bin/sh -n ingress-nginx
  • /etc/nginx
  • $ ls -l
  • cat nginx.conf
• 监听26379端口，应该是configMap中的data的26379
  • 从configMap中取出26379端口要转发的地址
• 转发到tcp-develop-redis-cluster-6379

# TCP services

        server {
                preread_by_lua_block {
                        ngx.var.proxy_upstream_name="tcp-develop-redis-cluster-6379";
                }

                listen                  26379;

                listen                  [::]:26379;

                proxy_timeout           600s;
                proxy_next_upstream     on;
                proxy_next_upstream_timeout 600s;
                proxy_next_upstream_tries   3;

                proxy_pass              upstream_balancer;

        }

测试

• telnet可以通过
• telnet 192.168.221.131 30009