三种调度粘性,主要根据官方文档说明:
NodeSelector(定向调度)、NodeAffinity(Node亲和性)、PodAffinity(Pod亲和性)。
1. nodeSelector
提供简单的pod部署限制,pod选择一个或多个node的label部署。
① 给node添加label
kubectl label nodes
② 为pod添加nodeSelector机制
apiVersion: v1 kind: Pod metadata: name: nginx labels: env: test spec: containers: - name: nginx image: nginx imagePullPolicy: IfNotPresent nodeSelector: disktype: ssd
③ 部署pod
2. nodeAffinity
该功能是nodeSelector的改进,现在处于beta阶段。
主要的改进有以下几点:
- 语法更多样(不仅支持“AND”,)
- 不仅可以指定硬条件,还支持软条件
- 支持pod亲和性
当nodeAffinity成熟的时候,nodeSelector会被废弃。
requiredDuringSchedulingIgnoredDuringExecution #硬性强制
preferredDuringSchedulingIgnoredDuringExecution #软性配置
IgnoredDuringExecution 表示 ,如果一个pod所在的节点 在Pod运行期间其标签发生了改变,不再符合该Pod的节点亲和性需求,则系统将忽略Node上Label的变化,该pod继续在该节点上运行。
如果同时设置了nodeSelector和nodeAffinity,则需要同时满足才能成为候选者node。
下面看一个例子:
① 该pod只部署在具有label kubernetes.io/e2e-az-name=e2e-az1,kubernetes.io/e2e-az-name=e2e-az2的node上;且会优先选择具有label another-node-label-key= another-node-label-value的node,当然如果没有满足该条件的node,该pod也会部署在其它node上。
② operator支持In, NotIn, Exists, DoesNotExist, Gt, Lt。可以使用NotIn和DoesNotExist实现node的反亲和性,或者使用pod taints与tolerations实现。
③ 如果设置了多个nodeSelectorTerms,则只需要匹配其中一个就可以成为候选者node。
④ 如果设置了多个matchExpressions,则需要全部匹配才能成为候选者node。
⑤ weight取值范围是1-100,对于有多个软条件的情况时,将匹配了改条件的weight相加,取最大的值为最优先候选者node。
# cat pods/pod-with-node-affinity.yaml
pods/pod-with-node-affinity.yaml
apiVersion: v1 kind: Pod metadata: name: with-node-affinity spec: affinity: nodeAffinity: requiredDuringSchedulingIgnoredDuringExecution: #hard条件必须匹配 nodeSelectorTerms: - matchExpressions: - key: kubernetes.io/e2e-az-name operator: In #支持In, NotIn, Exists, DoesNotExist, Gt, Lt values: - e2e-az1 - e2e-az2 preferredDuringSchedulingIgnoredDuringExecution: #soft条件优先匹配 - weight: 1 #取值范围1-100 preference: matchExpressions: - key: another-node-label-key operator: In values: - another-node-label-value containers: - name: with-node-affinity image: k8s.gcr.io/pause:2.0
3. Inter-pod affinity and anti-affinity (beta feature)
pod亲和性与反亲和性是根据pod的label挑选scheduler的候选者node,而不是根据node的label。
pod亲和性只在一个namespace生效,因为pod具有namespace,所以pod亲和性设置隐含了namespace。
topologyKey指示作用域,使用node的label的一个key值表示。
还可以使用一个namespaces列表限定schedulerr调度时查找的pod限定,namespaces放在labelSelector和topologyKey同一层,如:
podAntiAffinity: requiredDuringSchedulingIgnoredDuringExecution: - labelSelector: matchExpressions: - key: appname operator: In values: - dbpool-server topologyKey: kubernetes.io/hostname namespaces: #这样只会查找poa-ea和pletest下面的pod,而不是全部 - poa-ea - pletest
注意:Inter-pod affinity and anti-affinity需要消耗大量计算资源,会增加调度时间。如果node数量超过几百台的时候不建议使用。
注意:Pod反亲和性需要制定topologyKey
下面看一个例子:
① 出于安全考虑,requiredDuringSchedulingIgnoredDuringExecution的anti-affinity,topologyKey不允许为空;
② For requiredDuringSchedulingIgnoredDuringExecution pod anti-affinity, the admission controller LimitPodHardAntiAffinityTopology was introduced to limit topologyKey to kubernetes.io/hostname. If you want to make it available for custom topologies, you may modify the admission controller, or simply disable it.
③ For preferredDuringSchedulingIgnoredDuringExecution pod anti-affinity, empty topologyKey is interpreted as “all topologies” (“all topologies” here is now limited to the combination of kubernetes.io/hostname, failure-domain.beta.kubernetes.io/zone and failure-domain.beta.kubernetes.io/region).
pods/pod-with-pod-affinity.yaml
apiVersion: v1 kind: Pod metadata: name: with-pod-affinity spec: affinity: podAffinity: requiredDuringSchedulingIgnoredDuringExecution: - labelSelector: matchExpressions: - key: security operator: In values: - S1 topologyKey: failure-domain.beta.kubernetes.io/zone podAntiAffinity: preferredDuringSchedulingIgnoredDuringExecution: - weight: 100 podAffinityTerm: labelSelector: matchExpressions: - key: security operator: In values: - S2 topologyKey: kubernetes.io/hostname containers: - name: with-pod-affinity image: k8s.gcr.io/pause:2.0
4. 使用案例
需求:有一个web-server有3个实例,该web-server会使用到redis做为缓存。先需要将redis调度到和web-server同一个node。
① 部署redis,label app=store保证redis和web-server部署到相同的node
apiVersion: apps/v1 kind: Deployment metadata: name: redis-cache spec: selector: matchLabels: app: store replicas: 3 template: metadata: labels: app: store spec: affinity: podAntiAffinity: requiredDuringSchedulingIgnoredDuringExecution: - labelSelector: matchExpressions: - key: app operator: In values: - store topologyKey: "kubernetes.io/hostname" containers: - name: redis-server image: redis:3.2-alpine
② 部署web-server,与redis部署到一起,但是web-server之间不部署到一起。
apiVersion: apps/v1 kind: Deployment metadata: name: web-server spec: selector: matchLabels: app: web-store replicas: 3 template: metadata: labels: app: web-store spec: affinity: podAntiAffinity: requiredDuringSchedulingIgnoredDuringExecution: - labelSelector: matchExpressions: - key: app operator: In values: - web-store topologyKey: "kubernetes.io/hostname" podAffinity: requiredDuringSchedulingIgnoredDuringExecution: - labelSelector: matchExpressions: - key: app operator: In values: - store topologyKey: "kubernetes.io/hostname" containers: - name: web-app image: nginx:1.12-alpine
5. 参考资料
http://blog.51cto.com/newfly/2066630
https://kubernetes.io/docs/concepts/configuration/assign-pod-node/#affinity-and-anti-affinity