Node Selector - K8S标签调度

k8s本身就会根据资源等信息将pod分配到合适的node上，但有些场景下，我们需要更强的控制，比如云平台的租户只能使用购买的节点。有3种方式可以实现：node selector，affinity和node name。

Node Selector

Node Selector是指定pod分配到指定node上最简单的方法，使用Pod中的nodeSelector属性来实现。Node Selector会指定key-value pairs，pod会被分配到特定node上，该node具有所有指定key-value pairs对应labels。通常只有一对key-value。

下面命令为node设置label和查看node上label。

kubectl label nodes <node-name> <label-key>=<label-value>
kubectl get nodes --show-labels
kubectl describe node "nodename"

下面创建pod并指定node selector。但通常会使用Deployment等方式创建pod，同样也可以设置node selector.
kubectl label node k8s-n2 disktype=ssd #给k8s-n2节点增加标签

apiVersion: v1
kind: Pod
metadata:
  name: nginx
  labels:
    env: test
spec:
  containers:
  - name: nginx
    image: nginx
    imagePullPolicy: IfNotPresent
  nodeSelector:
    disktype: ssd

实际使用
kubectl label node cn-shenzhen.10.0.21.33 apptype=dwgtodfx-service
kubectl label node cn-shenzhen.10.0.21.33 apptype- #减去标签
nodeSelector:
apptype: @NODE_LABEL@

注意：当集群中不存在NodeSelector指定的Node，即使集群中还有其他可供使用的Node，这个Pod也不会被成功调度。

NodeSelector将会被继续使用，但最终会被废弃。

Affinity

相对于node selector，affinity更加灵活。除了node selector提供的label强制匹配外， affinity具有3个优势：1. 可以反向指定anti-affinity；2. 可以指定弱匹配，prefer，即使不匹配，也可能被分配； 3.可 提供node中pod之间的限制，不仅仅是node层级。

Affinity分为Node Affinity和Inter-pod Affinity.前者在使用node label进行选择，后者根据相同node上的pod label进行选择。

-Node Affinity

Node Affinity类似于Node Selector，但包括2种类型：requiredDuringSchedulingIgnoredDuringExecution和preferredDuringSchedulingIgnoredDuringExecution。前者表示强匹配，后者表示弱匹配。这2种类型都是pod schedule时起作用，运行时忽略。即pod分配之后，修改label不会影响已经分配的pod。后面计划提供requiredDuringSchedulingRequiredDuringExecution功能。

下面是node affinity示例。node必须具有label,其key为http://kubernetes.io/e2e-az-name，其value为e2e-az1或e2e-az2，同时，更倾向于具有label，其key为another-node-label-key， 其value为another-node-label-value。

kubectl label node cn-shenzhen.10.0.21.33 percona80-master.devops=true

pods:
  affinity:
    rules:
      nodeAffinity:
        requiredDuringSchedulingIgnoredDuringExecution:
          nodeSelectorTerms:
            - matchExpressions:
              - key: percona80-master.devops
                operator: In
                values:
                  - "true"

物业sass模板

apiVersion:  apps/v1
kind: Deployment
metadata:
  name: ${service}
  labels:
    app.kubernetes.io/name: ${service}
    app.kubernetes.io/version: 0.0.0
    app.kubernetes.io/instance: ${service}
    environment: ${env} #环境变量
spec:
  replicas: ${replicas} #副本数
  revisionHistoryLimit: 3
  selector:
    matchLabels:
      app.kubernetes.io/name: ${service}
  strategy:
    rollingUpdate:
      maxSurge: 25%
      maxUnavailable: 25%
    type: RollingUpdate
  template:
    metadata:
      labels:
        app.kubernetes.io/name: ${service}
        app.kubernetes.io/version: 0.0.0
        app.kubernetes.io/instance: ${service}
        micrometer-registry-prometheus: "true"
        logging: "false"
    spec:
      serviceAccountName: default
      dnsPolicy: Default
      imagePullSecrets:
        - name: gemdale-registry.cn-shenzhen.cr.aliyuncs.com-secret
      containers:
        - name: ${service}
          image: ${image}
          imagePullPolicy: IfNotPresent
          command:
            - java
          args:
            - -Duser.timezone=Asia/Shanghai
            - -XX:+UseContainerSupport
            - -XX:InitialRAMPercentage=80.0
            - -XX:MaxRAMPercentage=80.0
            - -jar
            - /data/var/www/service/${service}/${jar_name}.jar
          env:
            - name: HOST_IP
              valueFrom:
                fieldRef:
                  fieldPath: status.hostIP

            - name: WFWAPP
              value: wfw-applog

          volumeMounts:
            - mountPath: /data/appdata/
              name: appdata
            - mountPath: /data/config-repo/
              name: config-repo
            - mountPath: /data/logs/
              name: logs
            - mountPath: /mnt/hgfs/
              name: mnt-hgfs
          readinessProbe:
            tcpSocket:
              port: ${port}
            initialDelaySeconds: 30
            periodSeconds: 15
          ports:
            - containerPort: ${port}
              name: http
          resources:
            requests:
              memory: ${requests_memory} #内存资源下限 比如：500Mi、1Gi
              cpu: ${requests_cpu} #CPU资源下限，比如：0.1、0.25
            limits:
              memory: ${limit_memory} #内存资源上限 比如：800Mi、2Gi
              cpu: ${limit_cpu} #CPU资源上限，比如：0.5、1、2

      affinity:
        nodeAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
            nodeSelectorTerms:
              - matchExpressions:
                  - key: microservice
                    operator: In
                    values:
                      - "true"
      volumes:
        - hostPath:
            path: /data/appdata/
            type: DirectoryOrCreate
          name: appdata
        - hostPath:
            path: /data/config-repo/
            type: DirectoryOrCreate
          name: config-repo
        - hostPath:
            path: /data/logs/
            type: DirectoryOrCreate
          name: logs
        - hostPath:
            path: /mnt/hgfs/
            type: DirectoryOrCreate
          name: mnt-hgfs
---
apiVersion: v1
kind: Service
metadata:
  name: ${service}
  labels:
    app.kubernetes.io/name: ${service}
    app.kubernetes.io/version: 0.0.0
    app.kubernetes.io/instance: ${service}
    environment: ${env}
spec:
  type: ClusterIP
  ports:
    - name: http
      port: 80
      protocol: TCP
      targetPort: ${port}
    - name: https
      port: 443
      protocol: TCP
      targetPort: ${port}
  selector:
    app.kubernetes.io/name: ${service}
    app.kubernetes.io/instance: ${service}

智能审图模板

apiVersion: apps/v1
kind: Deployment  
metadata:  
  name: @APP_NAME@
  labels:  
    app: @APP_NAME@
spec:  
  replicas: @REPLICAS@
  revisionHistoryLimit: 10
  selector:  
    matchLabels:  
      app: @APP_NAME@
  template:  
    metadata:  
      labels:  
        app: @APP_NAME@
    spec:  
      containers:  
      - name: @APP_NAME@
        image: ${IMAGE}
        ports:  
        - containerPort: @targetPort@
          protocol: TCP  
        imagePullPolicy: IfNotPresent
        env:
        - name: protocol
          value: "http"
        - name: domainPrefix
          value: "://10.0.21.21"
        - name: app_env
          value: ":"
        - name: domainDelimiter
          value: "300"
        - name: domain
          value: "48"
        - name: nacos_namespace
          value: "c357bb7a-36a8-4880-a5ae-9778119688a5"
        - name: user_name
          value: "nacos"
        - name: password
          value: "Nacos@qaz"
        livenessProbe:
          failureThreshold: 3
          httpGet:
            path: /actuator
            port: @targetPort@
            scheme: HTTP
          initialDelaySeconds: 37
          periodSeconds: 10
          successThreshold: 1
          timeoutSeconds: 1
        readinessProbe:
          failureThreshold: 3
          httpGet:
            path: /actuator
            port: @targetPort@
            scheme: HTTP
          initialDelaySeconds: 3
          periodSeconds: 10
          successThreshold: 1
          timeoutSeconds: 1
        resources:
          limits:
            cpu: @LIMIT_CPU@
            memory: @LIMIT_MEMORY@
          requests:
            cpu: @REQUEST_CPU@
            memory: @REQUEST_MEMORY@          
        startupProbe:
          failureThreshold: 3
          httpGet:
            path: /actuator
            port: @targetPort@
            scheme: HTTP
          initialDelaySeconds: 17
          periodSeconds: 10
          successThreshold: 1
          timeoutSeconds: 1	        
          ######### 配置volume mount ###########
        volumeMounts:
          - name: volumn-logging-springboot
            mountPath: /gemdale/logs
      affinity:
        nodeAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
            nodeSelectorTerms:
              - matchExpressions:
                  - key: dwgtodfx-service
                    operator: In
                    values:
                      - "true"  
      volumes:
        - name: volumn-logging-springboot
          hostPath:
            type: DirectoryOrCreate
            path: /data/logs/@APP_NAME@
      ###############################
  
---  
apiVersion: v1  
kind: Service  
metadata:  
  name: @APP_NAME@
  namespace: default
  labels:  
    app: @APP_NAME@
spec:  
  ports:  
    - port: @port@
      targetPort: @targetPort@
      @NodePort@
  selector:  
    app: @APP_NAME@
  type: @PORT_TYPE@