kubernetes NetworkPolicy（防火墙）
开头语
写在前面：如有问题，以你为准，
目前24年应届生，各位大佬轻喷，部分资料与图片来自网络
内容较长，页面右上角目录方便跳转
概述
- 网络策略指的是 Pod 间的网络隔离策略，默认情况下是互通的。
- Pod 之间的互通是通过如下三个标识符的组合来辨识的：
- ① 其他被允许的 Pod（例外：Pod 无法阻塞对自身的访问）。
- ② 被允许的名称空间。
- ③ IP 组块（例外：与 Pod 运行所在的节点的通信总是被允许的， 无论 Pod 或节点的 IP 地址）
默认情况
如果命名空间没有配置任何NetworkPolicy，则允许所有流量进站和出站
网络策略(Network Policy)：是一个K8s资源，用于限制Pod出入流量，提供Pod级别和Namespace级别网络访问控制。
网络策略的应用场景：
- 应用程序间的访问控制，例如项目A不能访问项目B的Pod
- 开发环境命名空间不能访问测试环境命名空间Pod
- 当Pod暴露到外部时，需要做Pod白名单
- 多租户网络环境隔离
匹配策略
类似于安全组
当有一个命名空间有多个策略使用，只要其中一个策略是放通，其他策略都是拒绝，那也是放通
没有优先级的调整
只有允许策略，没有拒绝策略，（可以通过使用一个默认拒绝所有的策略和一个只开放策略来实现）
Pod 隔离和非隔离
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: networkpol-01
namespace: default
spec:
podSelector: # Pod 选择器
matchLabels:
app: nginx # 选中的 Pod 就被隔离起来了
policyTypes: # 策略类型
- Ingress # Ingress 入站规则、Egress 出站规则
- Egress
ingress: # 入站白名单，什么能访问我
- from:
- podSelector: # 选中的 Pod 可以访问 spec.matchLabels 中筛选的 Pod
matchLabels:
access: granted #有这个标签的pod才能访问被隔离的pod（上面的nginx）
ports: # 开放端口和协议
- protocol: TCP
port: 80
egress: # 出站白名单，我能访问什么
- to:
- podSelector: # spec.matchLabels 中筛选的 Pod 能访问的 Pod
matchLabels:
app: tomcat
- namespaceSelector:
matchLabels:
kubernetes.io/metadata.name: dev # spec.matchLabels 中筛选的 Pod 能访问 dev 命名空间下的所有
ports:
- protocol: TCP
port: 8080Namespace 隔离和非隔离
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: allow-port-from-namespace
namespace: my-app
spec:
podSelector: {}
policyTypes:
- Ingress
ingress:
- from:
- namespaceSelector:
matchLabels:
kubernetes.io/metadata.name: echo
# 拥有这个标签的namespace 可以访问my-app命名空间
ports:
- protocol: TCP
port: 9000拒绝操作
入站 yaml
默认拒绝所有入站流量，下面的yaml示例就是如此
# 默认拒绝所有入站流量
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: default-deny-ingress
namespace: default
spec:
podSelector: {}
policyTypes:
- Ingress
# 允许所有入站流量
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: allow-all-ingress
namespace: default
spec:
podSelector: {}
policyTypes:
- Ingress
ingress:
- {}出站 yaml
# 默认拒绝所有出站流量
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: default-deny-egress
spec:
podSelector: {}
policyTypes:
- Egress# 允许所有出站流量
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: allow-all-egress
spec:
podSelector: {}
policyTypes:
- Egress
egress:
- {}默认拒绝所有入站和所有出站流量
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: default-deny-all
spec:
podSelector: {} # 匹配本命名空间所有pod
policyTypes:
- Ingress
- Egress
# ingress 和 Egress 没有指定规则，则不允许任何流量进出pod整个 yaml 解析
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: test-network-policy
namespace: default # 配置NetworkPolicy的命名空间
spec:
podSelector:
matchLabels:
role: db # 在上面ns定义的命名空间下，符合这个labels的pod，才会被添加networkpolicy
policyTypes: # 定义类型
- Ingress #（入站）
- Egress #（出站）
ingress: # 入站的规则
- from: # 源地址，请求来自那个地址
- ipBlock: # ip地址
cidr: 172.17.0.0/16
except: # 除了这个以外
- 172.17.1.0/24
- namespaceSelector: # namespace 符合有下面这个标签的即可通过 （default命名空间下的pod）
matchLabels:
project: myproject
# kubernetes.io/metadata.name: test 指定某个命名空间下的所有pod
- podSelector: # 和上面隔离的命名空间的同一个命名空间下的pod，且有一下标签
matchLabels:
role: frontend
ports: # 开放的协议和端口
- protocol: TCP
port: 6379
egress: # 出站规则
- to: # 类似于上面的 from
- ipBlock: # 来自那个地址
cidr: 10.0.0.0/24
- podSelector: # spec.matchLabels 中筛选的 Pod 能访问的 Pod
matchLabels:
app: tomcat
- namespaceSelector:
matchLabels:
kubernetes.io/metadata.name: dev # spec.matchLabels 中筛选的 Pod 能访问 dev 命名空间下的所有
ports: # 开放协议端口
- protocol: TCP
port: 5978- 隔离 default 名字空间下 role=db 的 Pod （如果它们不是已经被隔离的话）。
- （Ingress 规则）允许以下 Pod 连接到 default 名字空间下的带有 role=db 标签的所有 Pod 的 6379 TCP 端口：
-
- default 名字空间下带有 role=frontend 标签的所有 Pod
- 带有 project=myproject 标签的所有名字空间中的 Pod
- IP 地址范围为 172.17.0.0–172.17.0.255 和 172.17.2.0–172.17.255.255 （即，除了 172.17.1.0/24 之外的所有 172.17.0.0/16）
- （Egress 规则）允许 default 名字空间中任何带有标签 role=db 的 Pod 到 CIDR 10.0.0.0/24 下 5978 TCP 端口的连接。
多个from或同一个from多个规则
or 的逻辑
ingress:
- from:
- namespaceSelector:
matchLabels:
kubernetes.io/metadata.name: default
- podSelector:
matchLabels:
app: client1and
ingress:
- from:
- namespaceSelector:
matchLabels:
kubernetes.io/metadata.name: default
- from:
- podSelector:
matchLabels:
app: client1[root@master cks]# kubectl describe networkpolicy -n test ns-app-allow-app
Name: ns-app-allow-app
Namespace: test
Created on: 2023-11-01 07:00:50 -0400 EDT
Labels:
Annotations:
Spec:
PodSelector: app=web
Allowing ingress traffic:
To Port: (traffic allowed to all ports)
From:
NamespaceSelector: kubernetes.io/metadata.name=default
From:
PodSelector: run=client1
Not affecting egress traffic
Policy Types: Ingress and 的逻辑
ingress:
- from:
- namespaceSelector:
matchLabels:
kubernetes.io/metadata.name: default
podSelector:
matchLabels:
app: client1[root@master cks]# kubectl describe networkpolicy -n test ns-app-allow-app
Name: ns-app-allow-app
Namespace: test
Created on: 2023-11-01 07:00:50 -0400 EDT
Labels:
Annotations:
Spec:
PodSelector: app=web
Allowing ingress traffic:
To Port: (traffic allowed to all ports)
From:
NamespaceSelector: kubernetes.io/metadata.name=default
PodSelector: run=client1
Not affecting egress traffic
Policy Types: Ingress 实操
命名空间级别
限制入站
创建环境（ns：test）
[root@master cks]# kubectl create ns test
namespace/test created
[root@master cks]# kubectl get ns
NAME STATUS AGE
calico-apiserver Active 2d2h
calico-system Active 2d2h
default Active 271d
ingress-nginx Active 256d
kube-node-lease Active 271d
kube-public Active 271d
kube-system Active 271d
kubernetes-dashboard Active 246d
study Active 231d
test Active 5s
tigera-operator Active 2d2h
[root@master cks]# kubectl run busybox -n test --image=busybox:1.30 -- sleep 12h
pod/busybox created
[root@master cks]# kubectl run nginx -n test --image=nginx:1.17.1
pod/nginx created
[root@master cks]# kubectl get pod -n test
NAME READY STATUS RESTARTS AGE
busybox 1/1 Running 0 54s
nginx 1/1 Running 0 4s创建环境 ns：default
[root@master cks]# kubectl run busybox --image=busybox:1.30 -- sleep 12h
pod/busybox created
[root@master cks]# kubectl get pod -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
busybox 1/1 Running 0 33s 10.244.104.36 node2 进入 busybox pod 里面进行ping（default）
[root@master cks]# kubectl get pod
NAME READY STATUS RESTARTS AGE
busybox 1/1 Running 0 2m46s
[root@master cks]# kubectl get pod -n test
NAME READY STATUS RESTARTS AGE
busybox 1/1 Running 0 5m47s
nginx 1/1 Running 0 4m57s
[root@master cks]# kubectl get pod -n test -owide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
busybox 1/1 Running 0 5m54s 10.244.104.34 node2
nginx 1/1 Running 0 5m4s 10.244.104.35 node2
[root@master cks]# kubectl exec -it busybox /bin/sh
kubectl exec [POD] [COMMAND] is DEPRECATED and will be removed in a future version. Use kubectl exec [POD] -- [COMMAND] instead.
/ # ping 10.244.104.35
PING 10.244.104.35 (10.244.104.35): 56 data bytes
64 bytes from 10.244.104.35: seq=0 ttl=63 time=0.124 ms
^C
--- 10.244.104.35 ping statistics ---
1 packets transmitted, 1 packets received, 0% packet loss
round-trip min/avg/max = 0.124/0.124/0.124 ms
/ # ping 10.244.104.34
PING 10.244.104.34 (10.244.104.34): 56 data bytes
64 bytes from 10.244.104.34: seq=0 ttl=63 time=0.093 ms
64 bytes from 10.244.104.34: seq=1 ttl=63 time=0.076 ms
^C
--- 10.244.104.34 ping statistics ---
2 packets transmitted, 2 packets received, 0% packet loss
round-trip min/avg/max = 0.076/0.084/0.093 ms
/ # 编写 networkpolicy，让其他所有命名空间访问不了test命名空间下的pod
# 默认拒绝所有入站流量
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: test-deny-ingress
namespace: test
spec:
podSelector: {}
policyTypes:
- Ingress[root@master cks]# kubectl create -f np-test-web.yaml
[root@master cks]# kubectl get NetworkPolicy -n test
NAME POD-SELECTOR AGE
test-deny-ingress 2m7s
[root@master cks]# kubectl describe NetworkPolicy -n test
Name: test-deny-ingress
Namespace: test
Created on: 2023-11-01 04:00:34 -0400 EDT
Labels:
Annotations:
Spec:
PodSelector: (Allowing the specific traffic to all pods in this namespace)
Allowing ingress traffic:
(Selected pods are isolated for ingress connectivity)
Not affecting egress traffic
Policy Types: Ingress 测试
# defualt 下的 busybox
[root@master cks]# kubectl exec -it busybox /bin/sh
kubectl exec [POD] [COMMAND] is DEPRECATED and will be removed in a future version. Use kubectl exec [POD] -- [COMMAND] instead.
/ # ping 10.244.104.35
PING 10.244.104.35 (10.244.104.35): 56 data bytes# test 下的 busybox
[root@master cks]# kubectl get pod -owide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
busybox 1/1 Running 0 12m 10.244.104.36 node2
[root@master cks]# kubectl exec -it busybox /bin/sh -n test
kubectl exec [POD] [COMMAND] is DEPRECATED and will be removed in a future version. Use kubectl exec [POD] -- [COMMAND] instead.
/ # ping www.baidu.com
PING www.baidu.com (14.119.104.254): 56 data bytes
64 bytes from 14.119.104.254: seq=0 ttl=127 time=7.583 ms
64 bytes from 14.119.104.254: seq=1 ttl=127 time=6.822 ms
/ # ping 10.244.104.36
PING 10.244.104.36 (10.244.104.36): 56 data bytes
64 bytes from 10.244.104.36: seq=0 ttl=63 time=0.071 ms
64 bytes from 10.244.104.36: seq=1 ttl=63 time=0.074 ms 结果表明：test 下的 pod 没办法被defualt的pod 访问，但是它可以访问外部
限制出站
# 默认拒绝所有入站流量
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: test-deny-ingress
namespace: test
spec:
podSelector: {}
policyTypes:
- Ingress
- Egress[root@master cks]# kubectl apply -f np-test-web.yaml
[root@master cks]# kubectl describe NetworkPolicy -n test
Name: test-deny-ingress
Namespace: test
Created on: 2023-11-01 04:00:34 -0400 EDT
Labels:
Annotations:
Spec:
PodSelector: (Allowing the specific traffic to all pods in this namespace)
Allowing ingress traffic:
(Selected pods are isolated for ingress connectivity)
Allowing egress traffic:
(Selected pods are isolated for egress connectivity)
Policy Types: Ingress, Egress
[root@master cks]# kubectl exec -it busybox /bin/sh -n test
kubectl exec [POD] [COMMAND] is DEPRECATED and will be removed in a future version. Use kubectl exec [POD] -- [COMMAND] instead.
/ # ping www.baidu.com
^C
/ # ping 10.244.104.36
PING 10.244.104.36 (10.244.104.36): 56 data bytes
^C
--- 10.244.104.36 ping statistics ---
3 packets transmitted, 0 packets received, 100% packet loss pod 级别
限制单个test 下的web pod 的访问权限
限制入站
kubectl run nginx -n test --image=nginx:1.17.1 -l="app=web"
[root@master cks]# kubectl get pod -n test --show-labels
NAME READY STATUS RESTARTS AGE LABELS
busybox 1/1 Running 0 30m run=busybox
nginx 1/1 Running 0 37s app=webapiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: test-deny-ingress
namespace: test
spec:
podSelector:
matchLabels:
app: web
policyTypes:
- Ingress
ingress:
- from:
- namespaceSelector:
matchLabels:
kubernetes.io/metadata.name: test
# - podSelector: {} # 匹配本命名空间所有pod [root@master cks]# kubectl apply -f np-test-web.yaml
networkpolicy.networking.k8s.io/test-deny-ingress changed测试
[root@master cks]# kubectl get pod -n test -owide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
busybox 1/1 Running 0 34m 10.244.104.34 node2
nginx 1/1 Running 0 4m27s 10.244.104.37 node2
[root@master cks]# kubectl exec -it busybox /bin/sh -n test
kubectl exec [POD] [COMMAND] is DEPRECATED and will be removed in a future version. Use kubectl exec [POD] -- [COMMAND] instead.
/ # ping 10.244.104.37
PING 10.244.104.37 (10.244.104.37): 56 data bytes
64 bytes from 10.244.104.37: seq=0 ttl=63 time=0.148 ms
64 bytes from 10.244.104.37: seq=1 ttl=63 time=0.105 ms
^C
--- 10.244.104.37 ping statistics ---
2 packets transmitted, 2 packets received, 0% packet loss
round-trip min/avg/max = 0.105/0.126/0.148 ms
/ # exit
command terminated with exit code 127
[root@master cks]# kubectl exec -it busybox /bin/sh
kubectl exec [POD] [COMMAND] is DEPRECATED and will be removed in a future version. Use kubectl exec [POD] -- [COMMAND] instead.
/ # ping 10.244.104.37
PING 10.244.104.37 (10.244.104.37): 56 data bytes
^C
--- 10.244.104.37 ping statistics ---
3 packets transmitted, 0 packets received, 100% packet loss 其他案例
（双策略）允许其他命名空间访问指定命名空间的指定 pod
编写一个拒绝的yaml：只有本命名空间可以访问web pod
# 只允许 test 命名空间可以相互访问，拒绝其他命名空间的访问
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: test-deny-ingress
namespace: test
spec:
podSelector: {}
policyTypes:
- Ingress
ingress:
- from:
- namespaceSelector:
matchLabels:
kubernetes.io/metadata.name: test
# 或者不指定ingress部分，默认是拒绝再写一允许yaml
# 只允许test下带标签 app: web 的 pod 可以被其他命名空间访问
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: test-allow-ingress
namespace: test
spec:
podSelector:
matchLabels:
app: web
policyTypes:
- Ingress
ingress:
- from:
- namespaceSelector:{}[root@master cks]# kubectl get NetworkPolicy -n test
NAME POD-SELECTOR AGE
test-allow-ingress app=web 8s
test-deny-ingress 56m
[root@master cks]# kubectl get pod -n test -owide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
busybox 1/1 Running 0 68m 10.244.104.34 node2
nginx 1/1 Running 0 38m 10.244.104.37 node2
[root@master cks]# kubectl exec -it busybox /bin/sh -n test
kubectl exec [POD] [COMMAND] is DEPRECATED and will be removed in a future version. Use kubectl exec [POD] -- [COMMAND] instead.
/ # ping 10.244.104.37
PING 10.244.104.37 (10.244.104.37): 56 data bytes
64 bytes from 10.244.104.37: seq=0 ttl=63 time=0.098 ms
# default 命名空间访问效果
[root@master cks]# kubectl exec -it busybox /bin/sh
kubectl exec [POD] [COMMAND] is DEPRECATED and will be removed in a future version. Use kubectl exec [POD] -- [COMMAND] instead.
/ # ping 10.244.104.34
PING 10.244.104.34 (10.244.104.34): 56 data bytes
--- 10.244.104.34 ping statistics ---
2 packets transmitted, 0 packets received, 100% packet loss
/ # ping 10.244.104.37
PING 10.244.104.37 (10.244.104.37): 56 data bytes
64 bytes from 10.244.104.37: seq=0 ttl=63 time=0.118 ms pod之间调用
同一命名空间下pod之间的调用
# 只允许 test 命名空间中的带有run:client1 可以访问
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: ns-app-allow-app
namespace: test
spec:
podSelector:
matchLabels:
run: web
policyTypes:
- Ingress
ingress:
- from:
- podSelector:
matchLabels:
run: client1 [root@master cks]# kubectl create -f np-test-web-client1.yaml
networkpolicy.networking.k8s.io/ns-app-allow-app created
[root@master cks]# kubectl get NetworkPolicy -n test
NAME POD-SELECTOR AGE
ns-app-allow-app run=web 5s
test-allow-ingress app=web 2m26s
test-deny-ingress 2m33s
[root@master cks]# kubectl get pod -n test -owide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
busybox 1/1 Running 0 156m 10.244.104.34 node2
client1 1/1 Running 0 2m20s 10.244.166.157 node1
nginx 1/1 Running 0 126m 10.244.104.37 node2
[root@master cks]# kubectl exec -it busybox /bin/sh -n test
kubectl exec [POD] [COMMAND] is DEPRECATED and will be removed in a future version. Use kubectl exec [POD] -- [COMMAND] instead.
/ # ping 10.244.166.157
PING 10.244.166.157 (10.244.166.157): 56 data bytes
[root@master cks]# kubectl exec -it client1 /bin/sh -n test
kubectl exec [POD] [COMMAND] is DEPRECATED and will be removed in a future version. Use kubectl exec [POD] -- [COMMAND] instead.
/ # ping 10.244.166.157
PING 10.244.166.157 (10.244.166.157): 56 data bytes
64 bytes from 10.244.166.157: seq=0 ttl=64 time=0.703 ms
64 bytes from 10.244.166.157: seq=1 ttl=64 time=0.037 ms 不同命名空间下pod之间的调用
# 只允许 default 命名空间中的带有run:client1 可以访问
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: ns-app-allow-app
namespace: test
spec:
podSelector:
matchLabels:
run: web
policyTypes:
- Ingress
ingress:
- from:
- namespaceSelector:
matchLabels:
name: default
podSelector:
matchLabels:
run: client1 测试
[root@master cks]# kubectl get pod -n test -owide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
busybox 1/1 Running 0 175m 10.244.104.34 node2
client1 1/1 Running 0 21m 10.244.166.157 node1
nginx 1/1 Running 0 145m 10.244.104.37 node2
[root@master cks]# kubectl exec -it client1 /bin/sh -n test
kubectl exec [POD] [COMMAND] is DEPRECATED and will be removed in a future version. Use kubectl exec [POD] -- [COMMAND] instead.
/ # ping 10.244.104.37
PING 10.244.104.37 (10.244.104.37): 56 data bytes
--- 10.244.104.37 ping statistics ---
2 packets transmitted, 0 packets received, 100% packet loss
[root@master cks]# kubectl exec -it client1 /bin/sh
kubectl exec [POD] [COMMAND] is DEPRECATED and will be removed in a future version. Use kubectl exec [POD] -- [COMMAND] instead.
/ # ping 10.244.104.37
PING 10.244.104.37 (10.244.104.37): 56 data bytes
64 bytes from 10.244.104.37: seq=0 ttl=63 time=0.128 ms
64 bytes from 10.244.104.37: seq=1 ttl=63 time=0.090 ms 所有命名空间的指定pod
# 允许所有命名空间中的带有run:client1 可以访问
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: ns-app-allow-app
namespace: test
spec:
podSelector:
matchLabels:
run: web
policyTypes:
- Ingress
ingress:
- from:
- namespaceSelector: {}
podSelector:
matchLabels:
run: client1