kubernetes API 访问控制
1.Authentication(认证):认证方式现共有8种,可以启用一种或多种认证方式,只要有一种认证方式通过,就不再 进行其它方式的认证。通常启用X509 Client Certs和Service Accout Tokens两种认证方式。
Kubernetes集群有两类用户:由Kubernetes管理的Service Accounts (服务账户)和 (Users Accounts) 普通账户。 k8s中账号的概念不是我们理解的账号,它并不真的存在, 它只是形式上存在。
2.Authorization(授权):必须经过认证阶段,才到授权请求,根据所有授权策略匹配请求资源属性,决定允许或拒 绝请求。授权方式现共有6种,AlwaysDeny、AlwaysAllow、ABAC、RBAC、Webhook、 Node。默认集群强制开启RBAC。
3.Admission Control(准入控制): 用于拦截请求的一种方式,运行在认证、授权之后,是权限认证链上的最后一环,对请求 API资源对象进行修改和校验。
访问k8s的API Server的客户端主要分为两类:
• kubectl : 用户家目录中的 .kube/config 里面保存了客户端访问API Server的密钥相关信息, 这样当用kubectl访问k8s时,它就会自动读取该配置文件,向API Server发起认证,然后完成操作请求。
• pod: Pod中的进程需要访问API Server,如果是人去访问或编写的脚本去访问,这类访问 使用的账号为:UserAccount;而Pod自身去连接API Server时,使用的账号是: ServiceAccount,生产中后者使用居多。
• kubectl向apiserver发起的命令,采用的是http方式,其实就是对URL发起增删改查的操作。
kubectl proxy --port=8888 &
curl http://localhost:8888/api/v1/namespaces/default
curl http://localhost:8888/apis/apps/v1/namespaces/default/deployments
以上两种api的区别是:
• api它是一个特殊链接,只有在核心v1群组中的对象才能使用。
• apis 它是一般API访问的入口固定格式名。
UserAccount与serviceaccount:
• 用户账户是针对人而言的。 服务账户是针对运行在 pod 中的进程而言的。
• 用户账户是全局性的。 其名称在集群各 namespace 中都是全局唯一的,未来的用户资源不会做 namespace 隔离, 服务账户是 namespace 隔离的。
• 通常情况下,集群的用户账户可能会从企业数据库进行同步,其创建需要特殊权限,并且涉及到复杂的业务流程。 服务账户创建的目的是为了更轻量,允许集群用户为了具体的任务创建服务账户 ( 即权限最小化原则 )。
kubectl create serviceaccount admin serviceaccount/admin created
创建serviceaccount
kubectl describe sa admin
此时k8s为用户自动生成认证信息,但没有授权
[kubeadm@server1 cm]$ kubectl get sa
NAME SECRETS AGE
default 1 15d
nfs-client-provisioner 1 2d2h
[kubeadm@server1 cm]$ kubectl create serviceaccount admin
serviceaccount/admin created
[kubeadm@server1 cm]$ kubectl get sa
NAME SECRETS AGE
admin 1 1s
default 1 15d
nfs-client-provisioner 1 2d2h
[kubeadm@server1 cm]$ kubectl describe sa admin
Name: admin
Namespace: default
Labels: <none>
Annotations: <none>
Image pull secrets: <none>
Mountable secrets: admin-token-vlffb
Tokens: admin-token-vlffb
Events: <none>
kubectl patch serviceaccount admin -p '{"imagePullSecrets": [{"name": "myregistrykey"}]}'
添加secrets到serviceaccount中
[kubeadm@server1 cm]$ kubectl patch serviceaccount admin -p '{"imagePullSecrets": [{"name": "myregistrykey"}]}'
serviceaccount/admin patched
[kubeadm@server1 cm]$ kubectl describe sa admin
Name: admin
Namespace: default
Labels: <none>
Annotations: <none>
Image pull secrets: myregistrykey
Mountable secrets: admin-token-vlffb
Tokens: admin-token-vlffb
Events: <none>
[kubeadm@server1 cm]$ vim pod.yml
[kubeadm@server1 cm]$ cat pod.yml
apiVersion: v1
kind: Pod
metadata:
name: mypod
spec:
containers:
- name: game2048
image: reg.red.org/private/game2048
serviceAccountName: admin
[kubeadm@server1 cm]$ kubectl apply -f pod.yml
pod/mypod created
[kubeadm@server1 cm]$ kubectl get pod
NAME READY STATUS RESTARTS AGE
mypod 1/1 Running 0 4s
将认证信息添加到serviceAccount中,要比直接在Pod指定imagePullSecrets要安全很多。
在默认sa default中,可以这样设置
[kubeadm@server1 cm]$ vim pod.yml
[kubeadm@server1 cm]$ cat pod.yml
apiVersion: v1
kind: Pod
metadata:
name: mypod
spec:
containers:
- name: game2048
image: reg.red.org/private/game2048
[kubeadm@server1 cm]$ kubectl patch serviceaccount default -p '{"imagePullSecrets": [{"name": "myregistrykey"}]}'
serviceaccount/default patched
[kubeadm@server1 cm]$ kubectl describe sa default
Name: default
Namespace: default
Labels: <none>
Annotations: <none>
Image pull secrets: myregistrykey
Mountable secrets: default-token-5qqxc
Tokens: default-token-5qqxc
Events: <none>
[kubeadm@server1 cm]$ kubectl apply -f pod.yml
pod/mypod created
[kubeadm@server1 cm]$ kubectl get pod
NAME READY STATUS RESTARTS AGE
mypod 1/1 Running 0 4s
[root@server1 ~]# cd /etc/kubernetes/pki/
[root@server1 pki]# ls
apiserver.crt apiserver.key ca.crt front-proxy-ca.crt front-proxy-client.key
apiserver-etcd-client.crt apiserver-kubelet-client.crt ca.key front-proxy-ca.key sa.key
apiserver-etcd-client.key apiserver-kubelet-client.key etcd front-proxy-client.crt sa.pub
[root@server1 pki]# openssl genrsa -out test.key 2048
Generating RSA private key, 2048 bit long modulus
.........................................................+++
.......................................................................................................................................+++
e is 65537 (0x10001)
[root@server1 pki]# openssl req -new -key test.key -out test.csr -subj "/CN=test"
[root@server1 pki]# openssl x509 -req -in test.csr -CA ca.crt -CAkey ca.key -CAcreateserial -out test.crt -days 365
Signature ok
subject=/CN=test
Getting CA Private Key
[root@server1 pki]# ll test.*
-rw-r--r-- 1 root root 973 Jul 4 18:28 test.crt
-rw-r--r-- 1 root root 883 Jul 4 18:28 test.csr
-rw-r--r-- 1 root root 1679 Jul 4 18:27 test.key
[root@server1 pki]# openssl x509 -in test.crt -text -noout
Certificate:
Data:
Version: 1 (0x0)
Serial Number:
bd:83:f0:28:d5:4c:88:26
Signature Algorithm: sha256WithRSAEncryption
Issuer: CN=kubernetes
Validity
Not Before: Jul 4 10:28:18 2020 GMT
Not After : Jul 4 10:28:18 2021 GMT
Subject: CN=test
Subject Public Key Info:
Public Key Algorithm: rsaEncryption
Public-Key: (2048 bit)
Modulus:
00:b4:d0:ae:81:52:c4:03:5e:3d:08:49:72:ae:7d:
cf:c0:d3:52:6c:50:6c:06:a2:b6:d6:2f:c3:ba:c2:
60:73:53:76:0e:66:b1:4e:c6:f4:71:30:31:b5:30:
84:14:c1:b6:8d:71:03:e1:f0:c0:79:34:c6:f4:26:
f7:41:77:9d:54:7c:fb:d5:6f:2c:06:9a:d2:ed:72:
f4:a2:b4:1f:6e:b1:a2:d6:77:fd:a8:3e:50:73:e6:
75:40:1c:39:de:a8:74:eb:01:69:98:a3:5f:9a:9d:
d4:24:51:20:41:36:21:c9:e4:c2:ca:ee:3e:e3:45:
1b:6b:90:4a:07:9b:c7:16:f3:7c:c0:ee:d4:d5:0a:
55:e6:09:23:f4:42:03:c4:3a:da:15:90:d4:c6:6e:
80:32:1b:a2:e3:ae:b6:a1:e6:65:41:42:9b:1c:ee:
b9:34:c0:7e:8e:5b:5a:b0:da:f8:fb:d4:a3:26:5a:
a6:f5:28:9f:f1:3c:fd:29:c0:b6:c2:67:d4:09:81:
7b:10:63:41:92:51:3d:01:91:3a:99:f2:4f:3e:b1:
91:14:7d:85:11:0d:e8:bb:7f:2c:52:15:08:f9:58:
13:31:20:26:d0:60:39:66:32:73:f6:6e:de:fd:2b:
ad:ca:c2:fe:07:6d:84:72:db:d5:a9:12:9f:2f:e2:
97:cb
Exponent: 65537 (0x10001)
Signature Algorithm: sha256WithRSAEncryption
62:1c:6a:9a:1e:97:f1:2c:c1:08:61:f2:a0:bf:13:b7:9f:7a:
d5:c8:05:d0:7b:67:d5:04:1c:84:43:9f:a1:e2:0a:d0:92:48:
13:70:dc:94:1a:40:97:47:af:a5:89:47:b7:bf:44:b5:83:bc:
42:1a:b4:92:59:e9:96:97:1e:8d:81:39:c2:d4:f7:46:cc:29:
1a:75:59:6e:1e:b0:4e:b4:0b:3e:67:7e:c1:12:6b:45:33:68:
8a:e5:c4:ca:24:d5:7a:b8:ba:2c:4b:3c:87:72:c1:12:69:a5:
20:2b:6b:5a:29:3f:d1:f3:0a:f3:ec:81:82:9b:08:d3:bf:49:
53:49:a4:fb:ec:43:fc:0f:4c:20:42:b5:d6:17:05:8b:8e:11:
0d:cd:03:c5:f2:eb:61:e1:21:76:86:b6:e4:cc:10:e4:c5:a1:
93:ab:e6:99:2f:59:30:e1:25:3d:fc:14:db:e4:4e:61:93:0b:
e5:3c:8a:66:ce:81:9e:aa:32:00:a4:fb:41:e9:f8:45:81:73:
83:ca:0e:e3:3c:c7:47:71:4d:46:51:16:56:da:dd:aa:28:58:
15:75:18:3d:fa:57:37:a6:08:ec:da:9a:6c:86:50:cb:26:8e:
6c:7a:a4:c2:96:78:0a:90:f2:d7:25:9b:31:bf:98:65:f1:5f:
c9:16:d1:7e
[kubeadm@server1 pki]$ kubectl config set-credentials test --client-certificate=/etc/kubernetes/pki/test.crt --client-key=/etc/kubernetes/pki/test.key --embed-certs=true
User "test" set.
[kubeadm@server1 pki]$ kubectl config set-context test@kubernetes --cluster=kubernetes --user=test
Context "test@kubernetes" created.
[kubeadm@server1 pki]$ kubectl config view
apiVersion: v1
clusters:
- cluster:
certificate-authority-data: DATA+OMITTED
server: https://192.168.43.11:6443
name: kubernetes
contexts:
- context:
cluster: kubernetes
user: kubernetes-admin
name: kubernetes-admin@kubernetes
- context:
cluster: kubernetes
user: test
name: test@kubernetes
current-context: kubernetes-admin@kubernetes
kind: Config
preferences: {}
users:
- name: kubernetes-admin
user:
client-certificate-data: REDACTED
client-key-data: REDACTED
- name: test
user:
client-certificate-data: REDACTED
client-key-data: REDACTED
[kubeadm@server1 pki]$ kubectl config use-context test@kubernetes
Switched to context "test@kubernetes".
[kubeadm@server1 pki]$ kubectl get pod
Error from server (Forbidden): pods is forbidden: User "test" cannot list resource "pods" in API group "" in the namespace "default"
RBAC(Role Based Access Control):基于角色访问控制授权
• 允许管理员通过Kubernetes API动态配置授权策略。RBAC就是用户通过角色与权限进行关联。
• RBAC只有授权,没有拒绝授权,所以只需要定义允许该用户做什么即可。
• RBAC包括四种类型:Role、ClusterRole、RoleBinding、ClusterRoleBinding。
RBAC的三个基本概念:
• Subject:被作用者,它表示k8s中的三类主体, user, group, serviceAccount
• Role:角色,它其实是一组规则,定义了一组对 Kubernetes API 对象的操作权限。
• RoleBinding:定义了“被作用者”和“角色”的绑定关系。
Role 和 ClusterRole
• Role是一系列的权限的集合,Role只能授予单个namespace 中资源的访问权限。 • ClusterRole 跟 Role 类似,但是可以在集群中全局使用。
[kubeadm@server1 ~]$ mkdir rbac
[kubeadm@server1 ~]$ cd rbac/
[kubeadm@server1 rbac]$ vim role.yml
[kubeadm@server1 rbac]$ cat role.yml
kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
namespace: default
name: myrole
rules:
- apiGroups: [""]
resources: ["pods"]
verbs: ["get", "watch", "list", "create", "update", "patch", "delete"]
[kubeadm@server1 rbac]$ kubectl config use-context kubernetes-admin@kubernetes
Switched to context "kubernetes-admin@kubernetes".
[kubeadm@server1 rbac]$ kubectl apply -f role.yml
role.rbac.authorization.k8s.io/myrole created
[kubeadm@server1 rbac]$ kubectl config use-context test@kubernetes
Switched to context "test@kubernetes".
[kubeadm@server1 rbac]$ kubectl get pod
Error from server (Forbidden): pods is forbidden: User "test" cannot list resource "pods" in API group "" in the namespace "default"
[kubeadm@server1 rbac]$ kubectl get role
NAME CREATED AT
leader-locking-nfs-client-provisioner 2020-07-02T07:12:08Z
myrole 2020-07-04T10:43:29Z
[kubeadm@server1 rbac]$ kubectl describe role myrole
Name: myrole
Labels: <none>
Annotations: PolicyRule:
Resources Non-Resource URLs Resource Names Verbs
--------- ----------------- -------------- -----
pods [] [] [get watch list create update patch delete]
RoleBinding是将Role中定义的权限授予给用户或用户组。它包含一个subjects 列表(users,groups service accounts),并引用该Role。
• RoleBinding是对某个namespace 内授权,ClusterRoleBinding适用在集群范围内使用。
RoleBinding示例:
[kubeadm@server1 rbac]$ vim role.yml
[kubeadm@server1 rbac]$ cat role.yml
kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
namespace: default
name: myrole
rules:
- apiGroups: [""]
resources: ["pods"]
verbs: ["get", "watch", "list", "create", "update", "patch", "delete"]
---
kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: test-read-pods
namespace: default
subjects:
- kind: User
name: test
apiGroup: rbac.authorization.k8s.io
roleRef:
kind: Role
name: myrole
apiGroup: rbac.authorization.k8s.io
[kubeadm@server1 rbac]$ kubectl apply -f role.yml
role.rbac.authorization.k8s.io/myrole unchanged
rolebinding.rbac.authorization.k8s.io/test-read-pods created
[kubeadm@server1 rbac]$ kubectl config use-context admin@kubernetes
error: no context exists with the name: "admin@kubernetes"
[kubeadm@server1 rbac]$ kubectl config use-context test@kubernetes
Switched to context "test@kubernetes".
[kubeadm@server1 rbac]$ kubectl get pod
NAME READY STATUS RESTARTS AGE
nfs-client-provisioner-96649cd96-pntbp 1/1 Running 1 22h
== ClusterRole示例, 使用rolebinding绑定clusterRole==
[kubeadm@server1 rbac]$ vim role.yml
[kubeadm@server1 rbac]$ kubectl config use-context kubernetes-admin@kubernetes
Switched to context "kubernetes-admin@kubernetes".
[kubeadm@server1 rbac]$ cat role.yml
kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
namespace: default
name: myrole
rules:
- apiGroups: [""]
resources: ["pods"]
verbs: ["get", "watch", "list", "create", "update", "patch", "delete"]
---
kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: test-read-pods
namespace: default
subjects:
- kind: User
name: test
apiGroup: rbac.authorization.k8s.io
roleRef:
kind: Role
name: myrole
apiGroup: rbac.authorization.k8s.io
---
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: myclusterrole
rules:
- apiGroups: [""]
resources: ["pods"]
verbs: ["get", "watch", "list", "delete", "create", "update"]
- apiGroups: ["extensions", "apps"]
resources: ["deployments"]
verbs: ["get", "list", "watch", "create", "update", "patch", "delete"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: rolebind-myclusterrole
namespace: default
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: myclusterrole
subjects:
- apiGroup: rbac.authorization.k8s.io
kind: User
name: test
[kubeadm@server1 rbac]$ kubectl apply -f role.yml
role.rbac.authorization.k8s.io/myrole unchanged
rolebinding.rbac.authorization.k8s.io/test-read-pods unchanged
clusterrole.rbac.authorization.k8s.io/myclusterrole created
rolebinding.rbac.authorization.k8s.io/rolebind-myclusterrole created
[kubeadm@server1 rbac]$ kubectl config use-context test@kubernetes
Switched to context "test@kubernetes".
[kubeadm@server1 rbac]$ cd ../mainfest/
[kubeadm@server1 mainfest]$ kubectl apply -f deployment.yml
deployment.apps/deployment-myapp created
[kubeadm@server1 mainfest]$ kubectl get pod
NAME READY STATUS RESTARTS AGE
deployment-myapp-7449b5b68f-hmmd2 1/1 Running 0 5s
deployment-myapp-7449b5b68f-kvczp 1/1 Running 0 5s
deployment-myapp-7449b5b68f-l8lkf 1/1 Running 0 5s
deployment-myapp-7449b5b68f-ndxql 1/1 Running 0 5s
nfs-client-provisioner-96649cd96-pntbp 1/1 Running 1 22h
[kubeadm@server1 mainfest]$ kubectl get pod -n kube-system
Error from server (Forbidden): pods is forbidden: User "test" cannot list resource "pods" in API group "" in the namespace "kube-system"
[kubeadm@server1 mainfest]$ kubectl delete -f deploy
error: the path "deploy" does not exist
[kubeadm@server1 mainfest]$ kubectl delete -f deployment.yml
deployment.apps "deployment-myapp" deleted
创建clusterrolebinding
[kubeadm@server1 rbac]$ cat role.yml
kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
namespace: default
name: myrole
rules:
- apiGroups: [""]
resources: ["pods"]
verbs: ["get", "watch", "list", "create", "update", "patch", "delete"]
---
kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: test-read-pods
namespace: default
subjects:
- kind: User
name: test
apiGroup: rbac.authorization.k8s.io
roleRef:
kind: Role
name: myrole
apiGroup: rbac.authorization.k8s.io
---
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: myclusterrole
rules:
- apiGroups: [""]
resources: ["pods"]
verbs: ["get", "watch", "list", "delete", "create", "update"]
- apiGroups: ["extensions", "apps"]
resources: ["deployments"]
verbs: ["get", "list", "watch", "create", "update", "patch", "delete"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: rolebind-myclusterrole
namespace: default
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: myclusterrole
subjects:
- apiGroup: rbac.authorization.k8s.io
kind: User
name: test
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: clusterrolebinding-myclusterrole
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: myclusterrole
subjects:
- apiGroup: rbac.authorization.k8s.io
kind: User
name: test
[kubeadm@server1 rbac]$ kubectl apply -f role.yml
role.rbac.authorization.k8s.io/myrole unchanged
rolebinding.rbac.authorization.k8s.io/test-read-pods unchanged
clusterrole.rbac.authorization.k8s.io/myclusterrole unchanged
rolebinding.rbac.authorization.k8s.io/rolebind-myclusterrole unchanged
clusterrolebinding.rbac.authorization.k8s.io/clusterrolebinding-myclusterrole created
[kubeadm@server1 rbac]$ kubectl config use-context test@kubernetes
Switched to context "test@kubernetes".
[kubeadm@server1 rbac]$ kubectl config get-contexts
CURRENT NAME CLUSTER AUTHINFO NAMESPACE
kubernetes-admin@kubernetes kubernetes kubernetes-admin
* test@kubernetes kubernetes test
[kubeadm@server1 rbac]$ kubectl get pod -n kube-system
NAME READY STATUS RESTARTS AGE
coredns-698fcc7d7c-nsg7m 1/1 Running 1 23h
coredns-698fcc7d7c-qp2ht 1/1 Running 1 22h
etcd-server1 1/1 Running 11 15d
kube-apiserver-server1 1/1 Running 12 15d
kube-controller-manager-server1 1/1 Running 11 15d
kube-flannel-ds-amd64-99khf 1/1 Running 2 23h
kube-flannel-ds-amd64-gsszl 1/1 Running 6 7d21h
kube-flannel-ds-amd64-p2mqf 1/1 Running 6 7d21h
kube-proxy-4xlms 1/1 Running 14 8d
kube-proxy-gx7jc 1/1 Running 14 8d
kube-proxy-n58d5 1/1 Running 14 8d
kube-scheduler-server1 1/1 Running 11 15d
[kubeadm@server1 rbac]$ kubectl get namespaces
Error from server (Forbidden): namespaces is forbidden: User "test" cannot list resource "namespaces" in API group "" at the cluster scope
[kubeadm@server1 rbac]$ kubectl config use-context kubernetes-admin@kubernetes
Switched to context "kubernetes-admin@kubernetes".
[kubeadm@server1 rbac]$ kubectl get namespaces
NAME STATUS AGE
default Active 15d
ingress-nginx Active 6d23h
kube-node-lease Active 15d
kube-public Active 15d
kube-system Active 15d
服务账户准入控制器(Service account admission controller)
• 如果该 pod 没有 ServiceAccount 设置,将其 ServiceAccount 设为 default。
• 保证 pod 所关联的 ServiceAccount 存在,否则拒绝该 pod。
• 如果 pod 不包含 ImagePullSecrets 设置,那么 将 ServiceAccount 中的 ImagePullSecrets 信息添加到 pod 中。
• 将一个包含用于 API 访问的 token 的 volume 添加到 pod 中。
• 将挂载于 /var/run/secrets/kubernetes.io/serviceaccount 的 volumeSource 添加到 pod 下的每个容器中。
Token 控制器(Token controller)
• 检测服务账户的创建,并且创建相应的 Secret 以支持 API 访问。
• 检测服务账户的删除,并且删除所有相应的服务账户 Token Secret。
• 检测 Secret 的增加,保证相应的服务账户存在,如有需要,为 Secret 增加 token。
• 检测 Secret 的删除,如有需要,从相应的服务账户中移除引用。
服务账户控制器(Service account controller)
• 服务账户管理器管理各命名空间下的服务账户,并且保证每个活跃的命名空间下存在 一个名为 “default” 的服务账户
Kubernetes 还拥有“用户组”(Group)的概念:
• ServiceAccount对应内置“用户”的名字是:
• system:serviceaccount:
• 而用户组所对应的内置名字是: • system:serviceaccounts:
示例1:表示mynamespace中的所有ServiceAccount
subjects:
- kind: Group
name: system:s=
erviceaccounts:mynamespace
apiGroup: rbac.authorization.k8s.io
示例2:表示整个系统中的所有ServiceAccount
subjects:
- kind: Group
name: system:serviceaccounts
apiGroup: rbac.authorization.k8s.io
Kubernetes 还提供了四个预先定义好的 ClusterRole 来供用户直接使用:
• cluster-amdin
• admin
• edit
• view
kubectl get clusterrole
[kubeadm@server1 rbac]$ kubectl describe clusterrole cluster-admin role.yml
Name: cluster-admin
Labels: kubernetes.io/bootstrapping=rbac-defaults
Annotations: rbac.authorization.kubernetes.io/autoupdate: true
PolicyRule:
Resources Non-Resource URLs Resource Names Verbs
--------- ----------------- -------------- -----
*.* [] [] [*]
[*] [] [*]
Error from server (NotFound): clusterroles.rbac.authorization.k8s.io "role.yml" not found