CustomResourceDefinition简介:
在 Kubernetes 中一切都可视为资源,Kubernetes 1.7 之后增加了对 CRD 自定义资源二次开发能力来扩展 Kubernetes API,通过 CRD 我们可以向 Kubernetes API 中增加新资源类型,而不需要修改 Kubernetes 源码来创建自定义的 API server,该功能大大提高了 Kubernetes 的扩展能力。
当你创建一个新的CustomResourceDefinition (CRD)时,Kubernetes API服务器将为你指定的每个版本创建一个新的RESTful资源路径,我们可以根据该api路径来创建一些我们自己定义的类型资源。CRD可以是命名空间的,也可以是集群范围的,由CRD的作用域(scpoe)字段中所指定的,与现有的内置对象一样,删除名称空间将删除该名称空间中的所有自定义对象。customresourcedefinition本身没有名称空间,所有名称空间都可以使用。
- 目前扩展Kubernetes API的常用方式有3种:
- 使用CRD(CustomResourceDefinitions)自定义资源类型
- 开发自定义的APIServer并聚合至主API Server
- 及定制扩展API Server源码。这其中,CRD最为易用但限制颇多,自定义API Server更富于弹性但代码工作量偏大,而仅在必须添加新的核心类型才能确保专用的Kberneves集群功能正常,才应该定制系统源码
- CRD-->CRT-->CR
其中CRD与CRT一般由开发或服务供应商提供
CRD只是定义一个类型Kind,但实际把kind运行起来CR需要有Controller来对资源进行控制,所有只有定义CRD定义没有并没有实际意义,当然也可以通过定义现在kind来运行,比如deployment 通过定义 RC来运行
配置规范
apiVersion: apiextensions.k8s.io/v1 #API群组和版本
kind: CustomResourceDefinition #资源类别
metadata:
-name #资源名称
spec:
conversion
- 可以查看之前部署Calico创建的自定义CRD
[root@k8s-master ~]# kubectl api-resources #查看所有资源类型
NAME SHORTNAMES APIGROUP NAMESPACED KIND
...
bgpconfigurations crd.projectcalico.org false BGPConfiguration
bgppeers crd.projectcalico.org false BGPPeer
blockaffinities crd.projectcalico.org false BlockAffinity
clusterinformations crd.projectcalico.org false ClusterInformation
felixconfigurations crd.projectcalico.org false FelixConfiguration
globalnetworkpolicies crd.projectcalico.org false GlobalNetworkPolicy
globalnetworksets crd.projectcalico.org false GlobalNetworkSet
hostendpoints crd.projectcalico.org false HostEndpoint
ipamblocks crd.projectcalico.org false IPAMBlock
ipamconfigs crd.projectcalico.org false IPAMConfig
ipamhandles crd.projectcalico.org false IPAMHandle
ippools crd.projectcalico.org false IPPool
kubecontrollersconfigurations crd.projectcalico.org false KubeControllersConfiguration
networkpolicies crd.projectcalico.org true NetworkPolicy
networksets crd.projectcalico.org true NetworkSet
查看calico的yaml文件可以看到里面很多CRD的定义
[root@k8s-master plugin]# vim calico.yaml ... --- apiVersion: apiextensions.k8s.io/v1 kind: CustomResourceDefinition metadata: name: ippools.crd.projectcalico.org spec: ...... ... [root@k8s-master plugin]# kubectl get CustomResourceDefinition NAME CREATED AT bgpconfigurations.crd.projectcalico.org 2021-08-29T14:33:24Z bgppeers.crd.projectcalico.org 2021-08-29T14:33:24Z blockaffinities.crd.projectcalico.org 2021-08-29T14:33:24Z clusterinformations.crd.projectcalico.org 2021-08-29T14:33:24Z felixconfigurations.crd.projectcalico.org 2021-08-29T14:33:24Z globalnetworkpolicies.crd.projectcalico.org 2021-08-29T14:33:24Z globalnetworksets.crd.projectcalico.org 2021-08-29T14:33:24Z hostendpoints.crd.projectcalico.org 2021-08-29T14:33:24Z ipamblocks.crd.projectcalico.org 2021-08-29T14:33:24Z ipamconfigs.crd.projectcalico.org 2021-08-29T14:33:24Z ipamhandles.crd.projectcalico.org 2021-08-29T14:33:24Z ippools.crd.projectcalico.org 2021-08-29T14:33:24Z kubecontrollersconfigurations.crd.projectcalico.org 2021-08-29T14:33:24Z networkpolicies.crd.projectcalico.org 2021-08-29T14:33:24Z networksets.crd.projectcalico.org 2021-08-29T14:33:25Z
示例1: 创建自定义CRD
[root@k8s-master crd]# cat crd-v1-user.yaml apiVersion: apiextensions.k8s.io/v1 kind: CustomResourceDefinition metadata: name: users.auth.ilinux.io spec: group: auth.ilinux.io names: kind: User plural: users singular: user shortNames: - u scope: Namespaced #名称空间级别 versions: - served: true storage: true name: v1alpha1 #版本号 schema: openAPIV3Schema: type: object #对字段做限制 properties: spec: type: object properties: userID: type: integer #整形 minimum: 1 maximum: 65535 groups : type: array #列表 items: type: string email: type: string password: type: string format: password required: ["userID","groups"] [root@k8s-master crd]# kubectl apply -f crd-v1-user.yaml [root@k8s-master crd]# kubectl api-resources NAME SHORTNAMES APIGROUP NAMESPACED KIND bindings true Binding ... users u auth.ilinux.io true User
创造自定义CRD类型
[root@k8s-master crd]# cat user-cr-demo.yaml apiVersion: auth.ilinux.io/v1alpha1 kind: User metadata: name: admin namespace: default spec: userID: 1 email: [email protected] groups: - superusers - adminstrators password: ikubernetes.io [root@k8s-master crd]# kubectl apply -f user-cr-demo.yaml user.auth.ilinux.io/admin created [root@k8s-master crd]# kubectl get User NAME AGE admin 14s [root@k8s-master ~]# kubectl describe User admin Name: admin Namespace: default Labels:
Annotations: API Version: auth.ilinux.io/v1alpha1 Kind: User Metadata: Creation Timestamp: 2021-09-10T14:51:53Z Generation: 1 Managed Fields: API Version: auth.ilinux.io/v1alpha1 Fields Type: FieldsV1 fieldsV1: f:metadata: f:annotations: .: f:kubectl.kubernetes.io/last-applied-configuration: f:spec: .: f:email: f:groups: f:password: f:userID: Manager: kubectl-client-side-apply Operation: Update Time: 2021-09-10T14:51:53Z Resource Version: 2583010 Self Link: /apis/auth.ilinux.io/v1alpha1/namespaces/default/users/admin UID: 5af89454-e067-4f30-83b7-cc2ad82e3526 Spec: Email: [email protected] Groups: superusers adminstrators Password: ikubernetes.io User ID: 1 Events: - 以上定义的kind资源 没Controller并不能运行成实际对象,Controller的开发需要开发来完成
示例2: etcd Operator 部署 (该项目已不在维护)
Operator 项目地址:
https://github.com/operator-f...
https://github.com/coreos/etc...
https://github.com/coreos/etc...-
先安装RBAC 再安装etcd operator 再部署创建etcd集群
[root@k8s-master etcd-operator]# example/rbac/create_role.sh Creating role with ROLE_NAME=etcd-operator, NAMESPACE=default Warning: rbac.authorization.k8s.io/v1beta1 ClusterRole is deprecated in v1.17+, unavailable in v1.22+; use rbac.authorization.k8s.io/v1 ClusterRole clusterrole.rbac.authorization.k8s.io/etcd-operator created Creating role binding with ROLE_NAME=etcd-operator, ROLE_BINDING_NAME=etcd-operator, NAMESPACE=default Warning: rbac.authorization.k8s.io/v1beta1 ClusterRoleBinding is deprecated in v1.17+, unavailable in v1.22+; use rbac.authorization.k8s.io/v1 ClusterRoleBinding clusterrolebinding.rbac.authorization.k8s.io/etcd-operator created [root@k8s-master etcd-operator]# kubectl create -f example/deployment.yaml error: unable to recognize "example/deployment.yaml": no matches for kind "Deployment" in version "extensions/v1beta1" #deployment版本太老修改example/deployment.yaml [root@k8s-master etcd-operator]# cat example/deployment.yaml apiVersion: apps/v1 #版本 kind: Deployment metadata: name: etcd-operator spec: replicas: 1 selector: #添加字段 matchLabels: name: etcd-operator template: metadata: labels: name: etcd-operator spec: containers: - name: etcd-operator image: quay.io/coreos/etcd-operator:v0.9.4 command: - etcd-operator # Uncomment to act for resources in all namespaces. More information in doc/user/clusterwide.md #- -cluster-wide env: - name: MY_POD_NAMESPACE valueFrom: fieldRef: fieldPath: metadata.namespace - name: MY_POD_NAME valueFrom: fieldRef: fieldPath: metadata.name [root@k8s-master etcd-operator]# kubectl create -f example/deployment.yaml deployment.apps/etcd-operator created [root@k8s-master etcd-operator]# [root@k8s-master etcd-operator]# kubectl api-resources ... etcdclusters etcd etcd.database.coreos.com true EtcdCluster
-
- 部署创建etcd集群
[root@k8s-master etcd-operator]# cat example/example-etcd-cluster.yaml apiVersion: "etcd.database.coreos.com/v1beta2" kind: "EtcdCluster" metadata: name: "example-etcd-cluster" ## Adding this annotation make this cluster managed by clusterwide operators ## namespaced operators ignore it # annotations: # etcd.database.coreos.com/scope: clusterwide spec: size: 3 #集群数理 version: "3.2.13" [root@k8s-master etcd-operator]# kubectl apply -f example/example-etcd-cluster.yaml etcdcluster.etcd.database.coreos.com/example-etcd-cluster created [root@k8s-master etcd-operator]# kubectl get pod -o wide NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES etcd-operator-646cbffdb6-brbn6 1/1 Running 0 12m 192.168.51.58 k8s-node3
example-etcd-cluster-nc8pdgjrjr 1/1 Running 0 3m3s 192.168.51.59 k8s-node3 后面在加一个SVC就可以使用了