k8s自定义调度器

k8s可使用自定义使用调度器。
内置调度器算法可在预选算法查看，优选算法查看。
优选算法分数范围在0-10分。
常用的优选算法介绍：

least_requested分值计算方式 (cpu((capacity-sum(requested))*10/capacity) + memory((capacity-sum(requested))*10/capacity))/2
其cpu和menmory分值各占一半。
balanced_resource_allocation是选用cpu和内存使用最为接近的节点，基于least_requested分值的计算结果判定。

自定义调度器的构建

下载源码，并构建

git clone https://github.com/kubernetes/kubernetes.git
cd kubernetes
make

Dockerfile内容,k8s版本是1.13.6，和官方文档有些区别，Dockerfile做了些修改

FROM alpine
ADD ./_output/local/bin/linux/amd64/kube-scheduler /usr/local/bin/kube-scheduler

docker image构建

docker build -t gcr.io/custom-gcp-project/custom-kube-scheduler:1.0 .

更改tag，上传自建仓库，如：

docker tag gcr.io/custom-gcp-project/custom-kube-scheduler:1.0 192.168.10.2/custom-gcp-project/custom-kube-scheduler:1.0
docker push  192.168.10.2/custom-gcp-project/custom-kube-scheduler:1.0

构建pod文件

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: custom-admin
rules:
- apiGroups:
  - '*'
  resources:
  - '*'
  verbs:
  - '*'
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: custom-scheduler
  namespace: kube-system
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: custom-scheduler-as-kube-scheduler
subjects:
- kind: ServiceAccount
  name: custom-scheduler
  namespace: kube-system
roleRef:
  kind: ClusterRole
  name: custom-admin
  apiGroup: rbac.authorization.k8s.io

---

apiVersion: v1
kind: Pod
metadata:
  annotations:
    scheduler.alpha.kubernetes.io/critical-pod: ""
  creationTimestamp: null
  labels:
    component: kube-scheduler
    tier: control-plane
  name: custom-scheduler
  namespace: kube-system
spec:
  serviceAccountName: custom-scheduler
  containers:
  - command:
    - kube-scheduler
    - --address=0.0.0.0
    - --policy-config-file=/policy-file.conf
    - --leader-elect=false
    - --scheduler-name=custom-scheduler
    image: 192.168.10.2/custom-gcp-project/custom-kube-scheduler:1.0
    imagePullPolicy: IfNotPresent
    livenessProbe:
      failureThreshold: 8
      httpGet:
        path: /healthz
        port: 10251
        scheme: HTTP
      initialDelaySeconds: 15
      timeoutSeconds: 15
    name: custom-scheduler
    resources:
      requests:
        cpu: 100m
    volumeMounts:
    - mountPath: /policy-file.conf
      name: policy-config
      readOnly: true
  volumes:
  - hostPath:
      path: /home/kube/custom-scheduler/policy-file.conf
      type: FileOrCreate
    name:  policy-config

status: {}

如上，ClusterRole赋予权限过大不适合在生产环境中使用，volumes相关的配置须在pod运行的节点中有相应的文件，更好的方式应该是使用configmap。
policy-file.conf内容：

{
"kind" : "Policy",
"apiVersion" : "v1",
"predicates" : [
	{"name" : "PodFitsPorts"},
	{"name" : "PodFitsResources"},
	{"name" : "NoDiskConflict"},
	{"name" : "MatchNodeSelector"},
	{"name" : "HostName"}
	],
"priorities" : [
	{"name" : "LeastRequestedPriority", "weight" : 1},
	{"name" : "BalancedResourceAllocation", "weight" : 1},
	{"name" : "ServiceSpreadingPriority", "weight" : 1},
	{"name" : "EqualPriority", "weight" : 1}
	]
}

测试能否被调用

apiVersion: v1
kind: Pod
metadata:
  name: annotation-second-scheduler
  labels:
    name: multischeduler-example
spec:
  schedulerName: custom-scheduler
  containers:
  - name: pod-with-second-annotation-container
    image: k8s.gcr.io/pause:2.0

schedulerName意为使用指定调度器，默认使用default-scheduler。
ref：https://kubernetes.io/docs/tasks/administer-cluster/configure-multiple-schedulers/