K8S之Headless返回IP浅析

K8S之Headless返回IP浅析

简介

• 有时不需要或不想要负载均衡，以及单独的 Service IP。 遇到这种情况，可以通过指定 Cluster IP（spec.clusterIP）的值为 “None” 来创建 Headless Service。(这个service是没有IP)。因为没有ClusterIP，kube-proxy 并不处理此类服务，因为没有load balancing或 proxy 代理设置，在访问服务的时候回返回后端的全部的Pods IP地址，主要用于开发者自己根据pods进行负载均衡器的开发(设置了selector) 或 StatefulSet
• 这个选项允许开发人员自由寻找他们自己的方式，从而降低与 Kubernetes 系统的耦合性。 应用仍然可以使用一种自注册的模式和适配器，对其它需要发现机制的系统能够很容易地基于这个 API 来构建。对这类 Service 并不会分配 Cluster IP，kube-proxy 不会处理它们，而且平台也不会为它们进行负载均衡和路由。 DNS如何实现自动配置，依赖于Service是否定义了 selector
• 不同的pod通过域名进行通信

ClusterIP=None

# 含有selector
[root@k8s-master-1 headless]# cat headless-selector.yaml 
apiVersion: v1
kind: Service
metadata:
  name: nginx
spec:
  type: ClusterIP
  clusterIP: None   # clusterIP = None
  selector:
    app: nginx
  ports:
  - port: 88
    targetPort: 80
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx
spec:
  replicas: 2
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      name: nginx
      labels:
        app: nginx
    spec:
      restartPolicy: Always
      containers:
      - name: nginx
        image: nginx:latest
        imagePullPolicy: IfNotPresent
        ports:
        - name: http
          containerPort: 80

# 查看POD
[root@k8s-master-1 headless]# kubectl get pods
NAME                    READY   STATUS    RESTARTS   AGE
nginx-d89c7cdcb-ql9fz   1/1     Running   0          2m57s
nginx-d89c7cdcb-vlmlp   1/1     Running   0          2m57s

# 查看svc
[root@k8s-master-1 headless]# kubectl get svc
NAME         TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)   AGE
kubernetes   ClusterIP   10.0.0.1     <none>        443/TCP   159d
nginx        ClusterIP   None         <none>        88/TCP    73s

[root@k8s-master-1 headless]# kubectl describe svc nginx
Name:              nginx
Namespace:         default
Labels:            <none>
Annotations:       <none>
Selector:          app=nginx
Type:              ClusterIP
IP Families:       <none>
IP:                None
IPs:               None
Port:              <unset>  88/TCP
TargetPort:        80/TCP
Endpoints:         10.70.2.58:80,10.70.2.61:80
Session Affinity:  None
Events:            <none>

# 运行busybox
[root@k8s-master-1 headless]# kubectl run busybox -it --rm --image=busybox:1.28 -- sh
If you don't see a command prompt, try pressing enter.
/ # nslookup nginx.default.svc.cluster.local
Server:    10.0.0.10
Address 1: 10.0.0.10 kube-dns.kube-system.svc.cluster.local

Name:      nginx.default.svc.cluster.local
Address 1: 10.70.2.58 10-70-2-58.nginx.default.svc.cluster.local
Address 2: 10.70.2.61 10-70-2-61.nginx.default.svc.cluster.local

## 由于nginx.default.svc.cluster.local 返回的是svc后面的POD IP，而后端pod的88端口并没有通

/ # telnet nginx.default.svc.cluster.local 88   
telnet: can't connect to remote host (10.70.2.58): Connection refused
/ # telnet nginx.default.svc.cluster.local 80
^C
Console escape. Commands are:

ClusterIP!=None

# 含有selector,ClusterIP!=None
[root@k8s-master-1 headless]# cat headless-selector.yaml 
apiVersion: v1
kind: Service
metadata:
  name: nginx
spec:
  type: ClusterIP
#  clusterIP: None   # clusterIP = None
  selector:
    app: nginx
  ports:
  - port: 88
    targetPort: 80
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx
spec:
  replicas: 2
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      name: nginx
      labels:
        app: nginx
    spec:
      restartPolicy: Always
      containers:
      - name: nginx
        image: nginx:latest
        imagePullPolicy: IfNotPresent
        ports:
        - name: http
          containerPort: 80

# 查看POD
[root@k8s-master-1 headless]# kubectl get pods
NAME                    READY   STATUS    RESTARTS   AGE
nginx-d89c7cdcb-5fm2f   1/1     Running   0          46s
nginx-d89c7cdcb-pgqxj   1/1     Running   0          46s

# 查看SVC
[root@k8s-master-1 headless]# kubectl get svc
NAME         TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)   AGE
kubernetes   ClusterIP   10.0.0.1       <none>        443/TCP   160d
nginx        ClusterIP   10.0.218.131   <none>        88/TCP    97s
[root@k8s-master-1 headless]# kubectl describe svc nginx
Name:              nginx
Namespace:         default
Labels:            <none>
Annotations:       <none>
Selector:          app=nginx
Type:              ClusterIP
IP Families:       <none>
IP:                10.0.218.131
IPs:               10.0.218.131
Port:              <unset>  88/TCP
TargetPort:        80/TCP
Endpoints:         10.70.2.2:80,10.70.2.63:80
Session Affinity:  None
Events:            <none>

# 运行busybox
[root@k8s-master-1 headless]# kubectl run busybox -it --rm --image=busybox:1.28 -- sh
If you don't see a command prompt, try pressing enter.
/ # nslookup nginx.default.svc.cluster.local
Server:    10.0.0.10
Address 1: 10.0.0.10 kube-dns.kube-system.svc.cluster.local

Name:      nginx.default.svc.cluster.local
Address 1: 10.0.218.131 nginx.default.svc.cluster.local

## 由于nginx.default.svc.cluster.local 返回的是svc IP,故而88端口会被转发到后端nginx的80端口，而svc 80端口未对外开放故而无法正常访问

/ # telnet nginx.default.svc.cluster.local 88
e
HTTP/1.1 400 Bad Request
Server: nginx/1.21.5
Date: Fri, 25 Mar 2022 01:37:00 GMT
Content-Type: text/html
Content-Length: 157
Connection: close




400 Bad Request
nginx/1.21.5


Connection closed by foreign host

/ # telnet nginx.default.svc.cluster.local 80
telnet: can't connect to remote host (10.0.218.131): Connection refused

总结

1. 配置了selector：对定义selector的Headless Service，Endpoint 控制器在 API 中创建了 Endpoints 记录，并且修改 DNS 配置返回 A 记录（地址），通过这个地址直接到达Service的后端Pod上
2. 未配置selector：对没有定义 selector 的 Headless Service，Endpoint 控制器不会创建Endpoints记录
3. 我们在容器里面ping FQDN ，clusterIP!=None 解析出的地址是 service的 clusterip，headless service 解析出来的地址是 pod ip
4. 通过headless service 可以轻松找到statefulSet 的所有节点。特别是在部署集群的时候，很多服务需要配置节点信息来创建集群
5. statefulSet.spec.serviceName：当serviceName 配置成与headless service的Name 相同的时候，可以通过 {hostName}.{service-name}.{namespace}.svc.cluster.local 解析出节点IP。hostName 由 {statefulSet name}-{编号} 组成