Admission Webhook 简介 + 示例

1. 简介

本文只做简单介绍以及示例，更多信息请查看官网
官网链接：https://kubernetes.io/docs/reference/access-authn-authz/extensible-admission-controllers/

1.1 作用范围

当用户的请求到达kube-apiserver后

1.2 可选操作

-MutatingWebhookConfiguration：修改用户请求的配置
-ValidatingWebhookConfiguration：验证用户请求的配置是否合法
顺序：先进行修改MutatingWebhookConfiguration再进行验证ValidatingWebhookConfiguration

1.3 先决条件

k8s版本1.9.0及以上，开启功能admissionregistration.k8s.io/v1alpha1
开启功能MutatingWebhookConfiguration或ValidatingWebhookConfiguration
每个kube-apiserver实例需要在enable-admission-plugins里添加支持
后端server
webhook的本质是个http server，因此，需要用代码实现这么一个server，供kube-apiserver调用，推荐使用Deployment+Service发布后端服务
证书以及Secret
用于kube-apiserver与webhook通信时使用，生成证书并创建Secret供Deployment挂载
创建MutatingWebhookConfiguration或ValidatingWebhookConfiguration资源

1.4 参数说明

1.4.1 官网yaml模板

apiVersion: admissionregistration.k8s.io/v1
kind: ValidatingWebhookConfiguration
metadata:
  name: "pod-policy.example.com"
webhooks:
- name: "pod-policy.example.com"
  rules:
  - apiGroups:   [""]
    apiVersions: ["v1"]
    operations:  ["CREATE"]
    resources:   ["pods"]
    scope:       "Namespaced"
  clientConfig:
    service:
      namespace: "example-namespace"
      name: "example-service"
    caBundle: "Ci0tLS0tQk...<`caBundle` is a PEM encoded CA bundle which will be used to validate the webhook's server certificate.>...tLS0K"
  admissionReviewVersions: ["v1", "v1beta1"]
  sideEffects: None
  timeoutSeconds: 5
---
# Deprecated in v1.16 in favor of admissionregistration.k8s.io/v1
apiVersion: admissionregistration.k8s.io/v1beta1
kind: ValidatingWebhookConfiguration
metadata:
  name: "pod-policy.example.com"
webhooks:
- name: "pod-policy.example.com"
  rules:
  - apiGroups:   [""]
    apiVersions: ["v1"]
    operations:  ["CREATE"]
    resources:   ["pods"]
    scope:       "Namespaced"
  clientConfig:
    service:
      namespace: "example-namespace"
      name: "example-service"
    caBundle: "Ci0tLS0tQk...<`caBundle` is a PEM encoded CA bundle which will be used to validate the webhook's server certificate>...tLS0K"
  admissionReviewVersions: ["v1beta1"]
  timeoutSeconds: 5

注意：

webhooks.name是一个DNS子域名
当使用clientConfig.service时，server端的证书必须对..svc验证有效
默认超时时长：
admissionregistration.k8s.io/v1 10s
admissionregistration.k8s.io/v1beta1 30秒

1.4.2 Webhook request and response

1.4.2.1 Webhook request

POST请求
Content-Type: application/json
参数内容是AdmissionReview，参数内容见此标题链接，不进行展示

1.4.2.1 Webhook response

http状态码返回200
Content-Type: application/json
参数内容AdmissionReview
- uid：取request里的uid
- allowed：true or false

{
  "apiVersion": "admission.k8s.io/v1beta1",
  "kind": "AdmissionReview",
  "response": {
    "uid": "",
    "allowed": true
  }
}

2. 示例`MutatingWebhookConfiguration`

2.1 查看是否支持`admissionregistration.k8s.io/v1alpha1`

[root@10 ~]# kubectl api-versions | grep admissionregistration.k8s.io/v1beta1
admissionregistration.k8s.io/v1beta1

2.2 查看是否开启`MutatingWebhookConfiguration`或`ValidatingWebhookConfiguration`

[root@10 ~]# kubectl -n kube-system get po $(kubectl -n kube-system get po | grep apiserver | awk '{print $1}') -oyaml | grep enable-admission-plugins
    - --enable-admission-plugins=NodeRestriction

如果没有的话，需要enable

// 默认路径/etc/kubernetes/manifests/kube-apiserver.yaml
- --enable-admission-plugins=NodeRestriction,MutatingAdmissionWebhook,ValidatingAdmissionWebhook

2.3 创建mutatingWebhookConfiguration

vi编译名为node-pool-mutating-webhook-config.yaml的yaml，格式如下

apiVersion: admissionregistration.k8s.io/v1beta1
kind: MutatingWebhookConfiguration
metadata:
  name: node-pool-config
  labels:
    app: node-pool-config
webhooks:
- clientConfig:
    caBundle: _CA_PEM_B64
    url: https://_F5_ADDRESS:30043/caas/rest/openapi/nodes/pools/mutate
  failurePolicy: Ignore
  name: mutate.node-pool.harmonycloud.cn
  rules:
    - apiGroups:
      - "*"
      apiVersions:
      - v1
      operations:
      - CREATE
      resources:
      - pods
  namespaceSelector:
    matchLabels:
      node-pool-mutating-webhook: enabled

需要注意的是：
-caBundle是kube-apiserver访问自定义的webhook服务时需要的加密认证数据，这边我们先预留参数，在下一步生成证书里再替换_CA_PEM_B64

url
_F5_ADDRESS可以是serviceName.namespace，也可以是某个nginx-ingress或者f5的ip
端口默认为443，可自行修改
接口路径自行修改
rules
规则里是配置对什么样的资源进行监听，在这里我们配置的是监听v1版本的任意pod创建
namespaceSelector
配置需要监听的namespace

2.4 生成证书和`Secret`

2.4.1 如果没有ca证书，可以生成

脚本如下：

openssl req -nodes -new -x509  -days 36500 -keyout ca.key -out ca.crt -subj "/CN=Admission Controller Webhook CA/subjectAltName=IP.1=${masterIp}"

openssl genrsa -out webhook-server-tls.key 2048

echo subjectAltName = IP:${masterIp} > extfile.cnf

openssl req -new -key webhook-server-tls.key -subj "/CN=${masterIp}/subjectAltName=IP.1=${masterIp}" | openssl x509 -days 36500 -req -CA ca.crt -CAkey ca.key -CAcreateserial -extfile extfile.cnf -out webhook-server-tls.crt

kubectl -n kube-system create secret tls webhook-server-tls \
    --cert "webhook-server-tls.crt" \
    --key "webhook-server-tls.key"

ca_pem_b64="$(openssl base64 -A <"ca.crt")"
sed -e 's@${CA_PEM_B64}@'"$ca_pem_b64"'@g' <"node-pool-mutating-webhook-config.yaml"

2.4.2 如果已有证书

// 假设ca.crt路径位于/root/
[root@10 ~]# openssl base64 -A < "/root/ca.crt"
LS0tLS1CRUdJTiBDRVJUxxxxxx

可以使用这个内容替换掉node-pool-mutating-webhook-config.yaml里的_CA_PEM_B64

2.5 创建`Deployment`和`Service`

自行创建这两个资源

注意，Deployment里需要挂载前面创建的Secretwebhook-server-tls
_F5_ADDRESS需要替换为serviceName.namespace，或者是某个nginx-ingress或者f5的ip

2.6 验证效果

2.6.1 创建`MutatingWebhookConfiguration`

[root@10 ~]# kubectl create -f node-pool-mutating-webhook-config.yaml
mutatingwebhookconfiguration.admissionregistration.k8s.io/node-pool-config created

2.6.2 修改namespace

根据node-pool-mutating-webhook-config.yaml里的matchLabels，修改某个分区的metadata.labels属性

2.6.3 在该分区下创建pod

测试podbusybox.yaml如下

apiVersion: v1
kind: Pod
metadata:
  name: test-pod
  namespace: cicd
  labels:
    app: test-pod
spec:
  containers:
  - name: test-container
    image: busybox
    command: ['sh', '-c', 'echo Hello Harmonycloud! && sleep 3600']

执行创建命令

[root@10 ~]# kubectl create -f busybox.yaml
pod/test-pod created

至此，大功告成！