tags:
categories:
调度过程分为两部分,如果中间任何一步骤有错误,直接返回错误:
**Predicate(预选)**有一系列的算法可以使用:
注意:如果在predicate过程中没有合适的节点。pod会一直在pending状态,不断重试调度,直到有节点满足条件。经过这个步骤,如果有多个节点满足条件,就继续priorities过程
**Priorities(优选)**是按照优先级大小对节点排序
优先级由一系列键值对组成,键是该优先级项的名称,值是它的权重(该项的重要性)。这些优先级选项包括:
通过算法对所有的优先级项目和权重进行计算,得出最终的结果。上面只是常见的算法,还有很多算法可以到官网查阅。
apiVersion: v1
kind: Pod
metadata:
name: affinity
labels:
app: node-affinity-pod
spec:
containers:
- name: with-node-affinity
image: hub.qnhyn.com/library/myapp:v1
affinity:
# 指定亲和性为node亲和性
nodeAffinity:
# 指定为硬策略
requiredDuringSchedulingIgnoredDuringExecution:
nodeSelectorTerms:
# key就是node的label
# 这句话代表当前pod一定不能分配到k8s-node02节点上
- matchExpressions:
- key: kubernetes.io/hostname # 标签的键名kubectl get node --show-labels查看
operator: NotIn
values:
- k8s-node2
apiVersion: v1
kind: Pod
metadata:
name: affinity
labels:
app: node-affinity-pod
spec:
containers:
- name: with-node-affinity
image: hub.qnhyn.com/library/myapp:v1
affinity:
# 声明节点亲和性为软策略
nodeAffinity:
preferredDuringSchedulingIgnoredDuringExecution:
# 当前策略权重为1
- weight: 1
preference:
# [最好]能分配到label为source=k8s-node03的节点上
matchExpressions:
- key: source
operator: In
values:
- k8s-node03
apiVersion: v1
kind: Pod
metadata:
name: pod-3
labels:
app: pod-3
spec:
containers:
- name: pod-3
image: hub.qnhyn.com/library/myapp:v1
affinity:
# 配置一条pod亲和性策略
podAffinity:
# 配置为硬策略 kubectl get pod --show-labels labels是app=pod1的pod同一拓扑域
requiredDuringSchedulingIgnoredDuringExecution:
- labelSelector:
matchExpressions:
- key: app
operator: In
values:
- pod-1
topologyKey: kubernetes.io/hostname
# 配置一条pod反亲和性策略
podAntiAffinity:
# 配置为软策略
preferredDuringSchedulingIgnoredDuringExecution:
- weight: 1
podAffinityTerm:
labelSelector:
matchExpressions:
- key: app
operator: In
values:
- pod-2
topologyKey: kubernetes.io/hostname
亲和性/反亲和性调度策略比较
调度策略 | 匹配标签 | 操作符 | 拓扑域支持 | 调度目标 |
---|---|---|---|---|
nodeAffinity | 主机 | IN, NotIn, Exists, DoesNotExist, Gt, Lt | 否 | 指定主机 |
podAffinity | POD | IN, NotIn, Exists, DoesNotExist | 是 | POD与指定POD同一拓扑域 |
podAntiAffinity | POD | IN, NotIn, Exists, DoesNotExist | 是 | POD与指定POD不在同一拓扑域 |
key=value:effect
key:effect
# 设置污点
kubectl taint nodes node1 key1=value1:NoSchedule
# 例子
kubectl taint nodes k8s-node1 check=qnhyn:NoExecute
# 节点说明中,查找Taints字段
kubectl describe pod pod-name
# 去除污点 通过describe查看污点,然后把污点复制出来,按照如下格式在最后加一个-就好了
kubectl taint nodes node1 key1:NoSchedule-
# 例子:
kubectl taint nodes k8s-node1 check=qnhyn:NoExecute-
tolerations:
# 容忍key1-value1:NoSchedule的污点
# 且需要被驱逐时,可以再呆3600秒
- key: "key1"
operator: "Equal"
value: "value1"
effect: "NoSchedule"
# 用于描述当Pod需要被驱逐时可以在 Pod上继续保留运行的时间
tolerationSeconds: 3600
# 容忍key1-value1:NoExecute的污点
- key: "key1"
operator: "Equal"
value: "value1"
effect: "NoExecute"
# 容忍key2:NoSchedule的污点
- key:"key2"
operator: "Exists"
effect: "NoSchedule"
tolerations:
- operator: "Exists"
tolerations:
- key: "key"
operator: "Exists"
kubectl taint nodes Node-Name node-role.kubernetes.io/master=:PreferNoSchedule
apiVersion: apps/v1
kind: Deployment
metadata:
name: myweb
spec:
replicas: 7
selector:
matchLabels:
app: myweb
template:
metadata:
labels:
app: myweb
spec:
# 直接指定node名称 七个Pod全在k8s-node1上
nodeName: k8s-node1
containers:
- name: myweb
image: hub.qnhyn.com/library/myapp:v1
ports:
- containerPort: 80
apiVersion: apps/v1
kind: Deployment
metadata:
name: myweb
spec:
replicas: 7
selector:
matchLabels:
app: myweb
template:
metadata:
labels:
app: myweb
spec:
# nodeSplector
nodeSplector:
disk: ssd # 标签为硬盘类型为ssd的
containers:
- name: myweb
image: hub.qnhyn.com/library/myapp:v1
ports:
- containerPort: 80
Kubernetes作为一个分布式集群的管理工具,保证集群的安全性是其一个重要的任务。 API Server是集群内部各个组件通信的中介,也是外部控制的入口。所以Kubernetes的安全机制基本就是围绕保护API Server来设计的。Kubernetes 使用了认证(Authentication) 、鉴权(Authorization) 、准入控制(Admission Control)三步来保证API Server的安全
Authentication认证方式有以下几种:
两种类型
安全性说明
证书颁发
kubeconfig:既是集群的描述,又是集群认证信息的填充。
ServiceAccount
Secret 与SA的关系
kubectl get secret --all-namespaces
kubectl describe secret default-token-5gm9r --namespace=kube-system
上面认证过程,只是确认通信的双方都确认了对方是可信的,可以相互通信。而鉴权是确定请求方有哪些资源的权限。API Server目前支持以下几种授权策略(通过 API Server的启动参数"-authorization-mode"设置)
RBAC (Role-Based Access Control)基于角色的访问控制,在Kubernetes 1.5中引入,现行版本成为默认标准。相对其它访问控制方式,拥有以下优势:
RBAC的API资源对象说明
需要注意的是Kubenetes并不会提供用户管理,那么User. Group. ServiceAccount 指定的用户又是从哪里来的呢? Kubenetes 组件(kubectl、 kube-proxy) 或是其他自定义的用户向CA申请证书时,要提供一个证书请求文件。用户名admin, 组system:masters
{
"CN": "admin",
"hosts": [],
"key": {
"algo": "rsa",
"size": 2048
},
"names": [
{
"C": "CN",
"ST": "HangZhou",
"L": "XS",
"O": "system:masters",
"OU": "System"
}
]
}
kind: Role
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
namespace: default
name: pod-reader
rules:
- apiGroups: [""] # "" indicates the core API group 不写代表core的核心组
resources: ["pods"]
verbs: ["get","watch","list"] # 如果把这个role给某个用户,这个用户就可以在default的名称空间下获取、监听、列出pod
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
# "namespace" omitted since ClusterRole are not namespaced
name: secret-reader
rules:
- apiGroups: [""]
resources: ["secrets"]
verbs: ["get","watch","list"]
kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
name: read-pods
namespace: default
subjects:
- kind: User
name: jane
apiGroup: rbac.authorization.k8s.io
roleRef:
kind: Role
name: pod-reader
apiGroup: rbac.authorization.k8s.io
# This role binding allows "dave" to read secrets in the "development" namespace.
kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
name: read-secrets
namespace: development # This only grants permissions within the "development' namespace
subjects:
- kind: User
name: dave
apiGroup: rbac.authorization.k8s.io
roleRef:
kind: ClusterRole
name: secret-reader
apiGroup: rbac.authorization.k8s.io
# This cluster role binding allows anyone in the "manager" group to read secrets in any namespace
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
name: read-secrets-global
subjects:
- kind: Group
name: manager
apiGroup: rbac.authorization.k8s.io
roleRef:
kind: ClusterRole
name: secret-reader
apiGroup: rbac.authorization.k8s.io
GET /api/v1/namespaces/{namespace}/pods/{name}/log
kind: Role
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
namespace: default
name: pod-and-pod-logs-reader
rules:
- apiGroups: [""]
resources: ["pods","pods/log"]
verbs: ["get","list"]
# 先创建一个用户
useradd devuser
passwd devuser
mkdir -p /usr/local/install-k8s/cert
cd /usr/local/install-k8s/cert
mkdir devuser && cd devuser
# 创建访问证书请求
vim devuser-csr.json
{
"CN":"devuser",
"hosts":[],
"key": {
"algo": "rsa",
"size": 2048
},
"names": [
{
"C": "CN",
"ST": "BeiJing",
"L": "BeiJing",
"O": "k8s",
"OU": "System"
}
]
}
# 下载证书生成工具
wget https://pkg.cfssl.org/R1.2/cfssl_linux-amd64
wget https://pkg.cfssl.org/R1.2/cfssljson_linux-amd64
wget https://pkg.cfssl.org/R1.2/cfssl-certinfo_linux-amd64
chmod a+x *
mv cfssl_linux-amd64 /usr/local/bin/cfssl
mv cfssljson_linux-amd64 /usr/local/bin/cfssljson
mv cfssl-certinfo_linux-amd64 /usr/local/bin/cfssl-certinfo
cd /etc/kubernetes/pki/
# 创建证书请求和证书私钥
cfssl gencert -ca=ca.crt -ca-key=ca.key -profile=kubernetes /usr/local/install-k8s/cert/devuser/devuser-csr.json | cfssljson -bare devuser
# 设置集群参数
cd /usr/local/install-k8s/cert/devuser
export KUBE_APISERVER="https://192.168.1.10:6443"
kubectl config set-cluster kubernetes \
--certificate-authority=/etc/kubernetes/pki/ca.crt \
--embed-certs=true \
--server=${KUBE_APISERVER} \
--kubeconfig=devuser.kubeconfig
# 设置客户端认证参数
kubectl config set-credentials devuser \
--client-certificate=/etc/kubernetes/pki/devuser.pem \
--client-key=/etc/kubernetes/pki/devuser-key.pem \
--embed-certs=true \
--kubeconfig=devuser.kubeconfig
# 设置上下文参数 帮我们绑定到某个名称空间dev
# 创建名称空间dev
kubectl create namespace dev
kubectl config set-context kubernetes \
--cluster=kubernetes \
--user=devuser \
--namespace=dev \
--kubeconfig=devuser.kubeconfig
# 创建一个rolebinging绑定clusterrole 绑定到命名空间dev.admin可以在dev中为所欲为
kubectl create rolebinding devuser-admin-binding --clusterrole=admin --user=devuser --namespace=dev
mkdir -p /home/devuser/.kube/
cp devuser.kubeconfig /home/devuser/.kube/
chown devuser:devuser /home/devuser/.kube/devuser.kubeconfig
cd /home/devuser/.kube/ && mv devuser.kubeconfig config
# 切换上下文 让kubectl读取我们的配置信息
kubectl config use-context kubernetes --kubeconfig=config
# 然后在devuser用户下测试权限
kubectl get pod
kubectl run nginx --image=hub.qnhyn.com/library/myapp:v1
kubectl get pod --all-namespaces
NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota