如何对kubernetes scheduler进行二次开发
通过新增Predicates&Priorities Policies来扩展default scheduler
新增Predicate Policy
- predicate Interface
plugin/pkg/scheduler/algorithm/types.go:31
// FitPredicate is a function that indicates if a pod fits into an existing node.
// The failure information is given by the error.
type FitPredicate func(pod *v1.Pod, meta interface{}, nodeInfo *schedulercache.NodeInfo) (bool, []PredicateFailureReason, error)- Implement a predicate func
func PodFitsHostNew(pod *v1.Pod, meta interface{}, nodeInfo *schedulercache.NodeInfo) (bool, []algorithm.PredicateFailureReason, error) {
if len(pod.Spec.NodeName) == 0 {
return true, nil, nil
}
node := nodeInfo.Node()
if node == nil {
return false, nil, fmt.Errorf("node not found")
}
if pod.Spec.NodeName == node.Name {
return true, nil, nil
}
return false, []algorithm.PredicateFailureReason{ErrPodNotMatchHostName}, nil
}- register the custom predicate policy with a custom name
plugin/pkg/scheduler/algorithmprovider/defaults/defaults.go:47
func init() {
...
factory.RegisterAlgorithmProvider(factory.DefaultProvider, defaultPredicates(), defaultPriorities())
// Cluster autoscaler friendly scheduling algorithm.
factory.RegisterAlgorithmProvider(ClusterAutoscalerProvider, defaultPredicates(),
copyAndReplace(defaultPriorities(), "LeastRequestedPriority", "MostRequestedPriority"))
...
factory.RegisterFitPredicate("CustomPredicatePolicy", predicates.PodFitsHostNew)
...
} - rebuild kube-scheduler and restart with flag of
--policy-config-file
kube-scheduler xxxx --policy-config-file=/var/lib/kube-scheduler/policy.config
- the content of
--policy-config-filespecified file
/var/lib/kube-scheduler/policy.config
{
"kind" : "Policy",
"apiVersion" : "v1",
"predicates" : [
{"name" : "CustomPredicatePolicy"}
],
"priorities" : [
]
}新增Priority Policy
- Priority Interface
/Users/garnett/workspace/go/src/k8s.io/kubernetes/plugin/pkg/scheduler/algorithm/types.go
// PriorityMapFunction is a function that computes per-node results for a given node.
type PriorityMapFunction func(pod *v1.Pod, meta interface{}, nodeInfo *schedulercache.NodeInfo) (schedulerapi.HostPriority, error)- Implement a predicate func
- register the custom predicate policy with a custom name
rebuild kube-scheduler and restart with flag of
--policy-config-filethe content of
--policy-config-filespecified file
/var/lib/kube-scheduler/policy.config
{
"kind" : "Policy",
"apiVersion" : "v1",
"predicates" : [
],
"priorities" : [
{"name" : "CumtomPriorityPolicy", "weight" : 1}
]
}新增custom scheduler，pod指定scheduler-name进行调度
- A custom scheduler can be written in any language and can be as simple or complex as you need.
- Specify the “scheduleName” in pod.spec
apiVersion: v1
kind: Pod
metadata:
name: nginx
labels:
app: nginx
spec:
schedulerName: my-scheduler
containers:
- name: nginx
image: nginx:1.10Here is a very simple example of a custom scheduler written in Bash that assigns a node randomly. Note that you need to run this along with kubectl proxy for it to work.
kubectl proxy --port=8001
#!/bin/bash
SERVER='localhost:8001'
while true;
do
for PODNAME in $(kubectl --server $SERVER get pods -o json | jq '.items[] | select(.spec.schedulerName == "my-scheduler") | select(.spec.nodeName == null) | .metadata.name' | tr -d '"')
;
do
NODES=($(kubectl --server $SERVER get nodes -o json | jq '.items[].metadata.name' | tr -d '"'))
NUMNODES=${#NODES[@]}
CHOSEN=${NODES[$[ $RANDOM % $NUMNODES ]]}
curl --header "Content-Type:application/json" --request POST --data '{"apiVersion":"v1", "kind": "Binding", "metadata": {"name": "'$PODNAME'"}, "target": {"apiVersion": "v1", "kind"
: "Node", "name": "'$CHOSEN'"}}' http://$SERVER/api/v1/namespaces/default/pods/$PODNAME/binding/
echo "Assigned $PODNAME to $CHOSEN"
done
sleep 1
done