11 . KubernetesRBAC认证及ServiceAccount、Dashboard

简介

Kubernetes RBAC角色权限控制

RBAC是基于角色的访问控制 (Role-Based Access Control) 在RBAC中，权限与角色相关联。Kubernetes 基于角色的访问控制使用rbac.authorization.k8s.io API组来实现权限控制，RBAC允许管理员通过Kubernetes API动态的配置权限策略。如果需要开启RBAC授权需要在apiserver组件中指定--authorization-mode=Node,RBAC

ServiceAccount

什么是service account? 顾名思义，相对于user account（比如：kubectl访问APIServer时用的就是user account），service account就是Pod中的Process用于访问Kubernetes API的account，它为Pod中的Process提供了一种身份标识。相比于user account的全局性权限，service account更适合一些轻量级的task，更聚焦于授权给某些特定Pod中的Process所使用。

Dashboard

Dashboard 是基于网页的 Kubernetes 用户界面。您可以使用 Dashboard 将容器应用部署到 Kubernetes 集群中，也可以对容器应用排错，还能管理集群本身及其附属资源。您可以使用 Dashboard 获取运行在集群中的应用的概览信息，也可以创建或者修改 Kubernetes 资源（如 Deployment，Job，DaemonSet 等等）。例如，您可以对 Deployment 实现弹性伸缩、发起滚动升级、重启 Pod 或者使用向导创建新的应用。

Kubernetes RBAC

APIServer是整个访问请求进入的网关接口，认证用于实现身份识别，授权用于权限检查.

K8s有两类账号

# 一种是用户账号user,一般是现实中人使用账号   
# 第二种是service账号，服务账号，Pod运行在K8s集群上，想访问K8s集群APIServer里面要用到的认证信息，账号密码等.  


API   
Request path   
http://localhost:6443/apis/apps/v1/namespaces/defaults/myapp-deplyoy/   
    
HTTP requests verb:   
 get,post,put,delete   
    
API requets verb:   
 get,list,create,update,patch,watch,proxy,redirect,delete,deletecollection   
    
Resource:   
 Subresource:   
 Namespace   
 API group   

创建Serviceaccount

# 快速创建一个yaml框架   
kubectl create serviceaccount mysa -o yaml --dry-run > serviceaccount.yaml   
    
kubectl get pods nginx-deployment-5dcb7cf7db-mjlsq  -o yaml --export   
    
kubectl create serviceaccount admin   
    
kubectl get sa   
NAME      SECRETS   AGE   
admin     1         4s   
default   1         4d8h   
mysa      1         3m45s   
    
# 这些认证不一定等于授权   
kubectl get secret   
NAME                  TYPE                                  DATA   AGE   
admin-token-9sxbn     kubernetes.io/service-account-token   3      35s   
default-token-kr5s2   kubernetes.io/service-account-token   3      4d8h   
mysa-token-rg974      kubernetes.io/service-account-token   3      4m16s   
    
cat pod-demo1.yaml   
apiVersion: v1   
kind: Pod   
metadata:   
 name: pod-sa-demo   
 namespace: default   
 labels:   
 app: myapp   
 tier: frontend   
 annotations:   
 youmen.com/create-by: "youmen admin"   
spec:   
 containers:   
 - name: myapp   
 image: ikubernetes/myapp:v1   
 ports:   
 - name: http   
 containerPort: 80   
kubectl describe pod pod-sa-demo |grep token   
 /var/run/secrets/kubernetes.io/serviceaccount from admin-token-9sxbn (ro)   
 admin-token-9sxbn:   
 SecretName:  admin-token-9sxbn   

Serviceaccount认证

我们正常敲kubectl get pods时候有哪些认证

kubectl config view   
apiVersion: v1   
clusters:  # 集群列表   
- cluster:   
 certificate-authority-data: DATA+OMITTED   
 server: https://192.168.78.7:6443   
 name: kubernetes   
contexts:  # 上下文列表   
- context:   
 cluster: kubernetes   
 user: kubernetes-admin   
 name: kubernetes-admin@kubernetes   
current-context: kubernetes-admin@kubernetes   
kind: Config   
preferences: {}   
users:  # 用户列表   
- name: kubernetes-admin   
 user:   
 client-certificate-data: REDACTED   
 client-key-data: REDACTED   

创建自己的一个用户

cd /etc/kubernetes/pki/   
# 创建私钥   
(umask 077;openssl genrsa -out youmen.key 2048)   
openssl  req -new -key youmen.key -out youmen.csr -subj "/CN=youmen"   
openssl x509 -req -in youmen.csr -CA ./ca.crt \   
-CAkey ./ca.key -CAcreateserial -out youmen.crt -days 365   
    
openssl x509 -in youmen.crt -text -noout   
Certificate:   
 Data:   
 Version: 1 (0x0)   
 Serial Number:   
 e6:1c:db:0b:a5:ca:01:db   
 Signature Algorithm: sha256WithRSAEncryption   
 Issuer: CN=kubernetes   
 Validity   
 Not Before: Dec 25 15:35:54 2019 GMT   
 Not After : Dec 24 15:35:54 2020 GMT   
 Subject: CN=youmen   
 Subject Public Key Info:   
 Public Key Algorithm: rsaEncryption   
 Public-Key: (2048 bit)   
    
kubectl config set-credentials youmen --client-certificate=./youmen.crt --client-key=./youmen.key --embed-certs=true   
    
kubectl config view   
apiVersion: v1   
clusters:   
- cluster:   
 certificate-authority-data: DATA+OMITTED   
 server: https://192.168.78.7:6443   
 name: kubernetes   
contexts:   
- context:   
 cluster: kubernetes   
 user: kubernetes-admin   
 name: kubernetes-admin@kubernetes   
current-context: kubernetes-admin@kubernetes   
kind: Config   
preferences: {}   
users:   
- name: kubernetes-admin   
 user:   
 client-certificate-data: REDACTED   
 client-key-data: REDACTED   
- name: youmen   
 user:   
 client-certificate-data: REDACTED   
 client-key-data: REDACTED   

创建一个上下文可以访问集群

kubectl config set-context youmen@kubernetes --cluster=kubernetes --user=youmen   
Context "youmen@kubernetes" created.   
[root@master pki]# kubectl config view   
apiVersion: v1   
clusters:   
- cluster:   
 certificate-authority-data: DATA+OMITTED   
 server: https://192.168.78.7:6443   
 name: kubernetes   
contexts:   
- context:   
 cluster: kubernetes   
 user: kubernetes-admin   
 name: kubernetes-admin@kubernetes   
- context:   
 cluster: kubernetes   
 user: youmen   
 name: youmen@kubernetes   
current-context: kubernetes-admin@kubernetes   
kind: Config   
preferences: {}   
users:   
- name: kubernetes-admin   
 user:   
 client-certificate-data: REDACTED   
 client-key-data: REDACTED   
- name: youmen   
 user:   
 client-certificate-data: REDACTED   
 client-key-data: REDACTED   

我们切换到youmen用户尝试一下

kubectl config use-context youmen@kubernetes   
Switched to context "youmen@kubernetes".   
    
kubectl get pods   
Error from server (Forbidden): pods is forbidden: User "youmen" cannot list resource "pods" in API group "" in the namespace "default"   

自己创建一个cluster集群

kubectl config set-cluster mycluster \   
--kubeconfig=/tmp/test.conf \   
--server="https://172.19.0.41:6443" \   
--certificate-authority=/etc/kubernetes/pki/ca.crt \   
--embed-certs=true   
    
 kubectl config view --kubeconfig=/tmp/test.conf   
apiVersion: v1   
clusters:   
- cluster:   
 certificate-authority-data: DATA+OMITTED   
 server: https://172.19.0.41:6443   
 name: mycluster   
contexts: null   
current-context: ""   
kind: Config   
preferences: {}   
users: null   

RBAC授权
之前我们切换用户get pods时候发现报错，还未赋予权限
k8s授权请求是http的请求
/apis/[GROUP]/[VERSION]/namespace/[NAMESPACE_NAME]/[KIND]/[OBJECT_ID]
k8s授权方式分为:
serviceccount和自己签证ca证书的账号，及签证ca的用户组(group)上（授权给这个组的权限)

role:

# 1. 允许的操作，如get,list等   
# 2. 允许操作的对象，如pod，svc等   

rolebinding:

# 将那个用户绑定到那个role或clusterrrole上   
# clusterrole: (集群角色)   
# clusterolebinding:(绑定到集群)  

如果使用rolebinding绑定到clusterrole上，表示绑定的用户只能用于当前namespace的权限

kubectl create role pods-reader --verb=get,list,watch --resource=pods  --dry-run -o yaml>role-demo.yaml   
    
cat role-demo.yaml   
apiVersion: rbac.authorization.k8s.io/v1   
kind: Role   
metadata:   
 creationTimestamp: null   
 name: pods-reader   
 namespace: default   
rules:   
- apiGroups:   
 - ""   
 resources:   
 - pods   
 verbs:   
 - get   
 - list   
 - watch   
    
kubectl get role   
NAME          AGE   
pods-reader   48s   
    
kubectl describe role pods-reader   
Name:         pods-reader   
Labels:          
Annotations:  kubectl.kubernetes.io/last-applied-configuration:   
 {"apiVersion":"rbac.authorization.k8s.io/v1","kind":\   
"Role","metadata":{"annotations":{},"creationTimestamp":null,"name":"pods-reader","nam...   
PolicyRule:   
 Resources  Non-Resource URLs  Resource Names  Verbs   
 ---------  -----------------  --------------  -----   
 pods       []                 []              [get list watch]   
    
# 绑定rolebinding   
kubectl create rolebinding youmen-read-pods --role=pods-reader --user=youmen   
kubectl create rolebinding youmen-read-pods --role=pods-reader --user=youmen --dry-run \   
-o yaml > rolebind-demo.yaml   

查看并验证绑定授权用户

kubectl describe rolebinding youmen-read-pods   
Name:         youmen-read-pods   
Labels:          
Annotations:     
Role:   
 Kind:  Role   
 Name:  pods-reader   
Subjects:   
 Kind  Name    Namespace   
 ----  ----    ---------   
 User  youmen   
    
kubectl config use-context youmen@kubernetes   
Switched to context "youmen@kubernetes".   
[root@master pki]# kubectl get pods   
NAME      READY   STATUS    RESTARTS   AGE   
myapp-0   1/1     Running   0          16h   
myapp-1   1/1     Running   0          16h   
myapp-2   1/1     Running   0          16h   
myapp-3   1/1     Running   0          16h   
    
# 但是如果换了一个明成空间就会失效   
kubectl get pods -n kube-system   
Error from server (Forbidden): pods is forbidden: User "youmen" cannot list resource "pods" in API group "" in the namespace "kube-system"   

创建clusterrole，可以访问全部的namespace

    
kubectl create clusterrole cluster-reader --verb=get,list,watch --resource=pods -o yaml --dry-run \   
> clusterrole-demo.yaml   
 useradd k8s   
 cp -rp .kube/ /home/k8s/   
 chown -R k8s:k8s /home/k8s/   
su - k8s   
kubectl config view   
apiVersion: v1   
clusters:   
- cluster:   
 certificate-authority-data: DATA+OMITTED   
 server: https://172.19.0.26:6443   
 name: kubernetes   
contexts:   
- context:   
 cluster: kubernetes   
 user: kubernetes-admin   
 name: kubernetes-admin@kubernetes   
- context:   
 cluster: kubernetes   
 user: youmen   
 name: youmen@kubernetes   
current-context: youmen@kubernetes   
kind: Config   
preferences: {}   
users:   
- name: kubernetes-admin   
 user:   
 client-certificate-data: REDACTED   
 client-key-data: REDACTED   
- name: youmen   
 user:   
 client-certificate-data: REDACTED   
 client-key-data: REDACTED   

cat cluterrole-demo.yaml   
apiVersion: rbac.authorization.k8s.io/v1   
kind: ClusterRole   
metadata:   
 name: cluster-reader   
rules:   
- apiGroups:   
 - ""   
 resources:   
 - pods   
 verbs:   
 - get   
 - list   
 - watch   
    
kubectl apply -f cluterrole-demo.yaml   
    
# 我们将之前的role删掉，然后会发现立马就没权限查看Pod了   
kubectl delete rolebinding youmen-read-pods   
kubectl get pods   
Error from server (Forbidden): pods is forbidden: User "youmen" cannot list resource "pods" in API group "" in the namespace "default"   
# 接下来我们使用clusterrolebinding更大的权限**   
kubectl create clusterrolebinding youmen-read-all-pods --clusterrole=cluster-reader --user=youmen   
kubectl create clusterrolebinding youmen-read-all-pods --clusterrole=cluster-reader --user=youmen --dry-run -o yaml \> clusterrolebinding-demo.yaml   
vim clusterrolebinding-demo.yaml   
apiVersion: rbac.authorization.k8s.io/v1beta1   
kind: ClusterRoleBinding   
metadata:   
 creationTimestamp: null   
 name: youmen-read-all-pods   
roleRef:   
 apiGroup: rbac.authorization.k8s.io   
 kind: ClusterRole   
 name: cluster-reader   
subjects:   
- apiGroup: rbac.authorization.k8s.io   
 kind: User   
 name: youmen   
    
kubectl get clusterrolebinding |grep youmen   
youmen-read-all-pods                                   2m18s   
kubectl describe clusterrolebinding youmen-read-all-pods   
Name:         youmen-read-all-pods   
Labels:          
Annotations:     
Role:   
 Kind:  ClusterRole   
 Name:  cluster-reader   
Subjects:   
 Kind  Name    Namespace   
 ----  ----    ---------   
 User  youmen   

验证clusterrolebinding

kubectl get pods -n kube-system   
NAME                             READY   STATUS    RESTARTS   AGE   
coredns-6955765f44-5jlbk         1/1     Running   1          7d17h   
coredns-6955765f44-xnr7f         1/1     Running   1          7d17h   
etcd-master                      1/1     Running   1          7d17h   
kube-apiserver-master            1/1     Running   1          7d17h   
    
kubectl delete clusterrolebinding youmen-read-all-pods   
# 我们将它删除,在创建一个新的clusterrolebinding   
kubectl create rolebinding youmen-read-pods \   
--clusterrole=cluster-reader \   
--user=youmen --dry-run -o yaml > rolebinding-clusterrole.demo.yaml   
    
cat rolebinding-clusterrole.demo.yaml   
apiVersion: rbac.authorization.k8s.io/v1   
kind: RoleBinding   
metadata:   
 creationTimestamp: null   
 name: youmen-read-pods   
 namespace: default   
roleRef:   
 apiGroup: rbac.authorization.k8s.io   
 kind: ClusterRole   
 name: cluster-reader   
subjects:   
- apiGroup: rbac.authorization.k8s.io   
 kind: User   
 name: youmen   
kubectl apply -f rolebinding-clusterrole.demo.yaml   
    
kubectl describe rolebinding youmen-read-pods   
Name:         youmen-read-pods   
Labels:          
Annotations:  kubectl.kubernetes.io/last-applied-configuration:   
 {"apiVersion":"rbac.authorization.k8s.io/v1","kind":"RoleBinding",\   
"metadata":{"annotations":{},"creationTimestamp":null,"name":"youmen-rea...   
Role:   
 Kind:  ClusterRole   
 Name:  cluster-reader   
Subjects:   
 Kind  Name    Namespace   
 ----  ----    ---------   
 User  youmen   
    
kubectl get pods   
NAME      READY   STATUS    RESTARTS   AGE   
myapp-0   1/1     Running   0          17h   
myapp-1   1/1     Running   0          17h   
myapp-2   1/1     Running   0          17h   
myapp-3   1/1     Running   0          17h   
# 我们可以发现服务被降级了   
[k8s@master ~]$ kubectl get pods -n kube-system   
Error from server (Forbidden): pods is forbidden: User "youmen" cannot list resource "pods" in API group "" in the namespace "kube-system"   

flannel控制

kubectl get pods -n kube-system kube-flannel-ds-amd64-jw4xk -o yaml   

Dashboard

Kubernetes集群的管理方式:

1. 命令式: create,run,expose,delete,..

2. 命令式配置文件: create -f /PATH/TO/RESOURCE_CONFGURATION_FILE ，delete -f ,replace -f

3. 声明式配置文件: apply -f,patch

4. 部署
   kubectl apply -f https://raw.githubusercontent.com/kubernetes/dashboard/v2.0.0-beta8/aio/deploy/recommended.yaml

5. 将Service改为NodePort

6. 认证

• 认证时账号必须为ServiceAccount,被dashboard pod拿来由kubernetes进行认证.
  token:
  1. 创建ServiceAccount,根据其管理目标,使用rolebinding或clusterrolebinding绑定至合理role或clusterrole；
  2. 获取此ServiceAccount的secret，查看secret的详细信息，其中就有token；

config: （把ServiceAccount的token封装为kubeconfig文件）
1. 创建ServiceAccount,根据其管理目标,使用rolebinding或clusterrolebinding绑定至合理role或clusterrole；
2.kubectl get secret |awk '/^ServiceAccount/{print $1}'
DEF_NS_ADMIN_TOKEN=$(kubectl get secret def-ns-admin-token-mnpph -o jsonpath={.data.token} |base64 -d )
3. 生成kubeconfig文件
kubectl config set-cluster --kubeconfig=/PATH/TO/SOMEFILE
kubectl config set-credentials NAME --token=$KUBE_TOKEN --kubeconfig=/PATH/TO/SOMEFILE
kubectl config set-context
kubectl config use-context

# 我们通过打补丁实现NodePort外部访问k8sUI  (不建议)  
kubectl patch -n kubernetes-dashboard svc kubernetes-dashboard -p '{"spec":{"type":"NodePort"}}'   
service/kubernetes-dashboard patched   
[root@master dashboard]# kubectl get svc -n kubernetes-dashboard   
NAME                        TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)         AGE   
dashboard-metrics-scraper   ClusterIP   10.96.191.177           8000/TCP        27m   
kubernetes-dashboard        NodePort    10.96.123.181           443:32630/TCP   27m   
# 此时访问需要用Pod所在节点的IP，但是因为证书问题，需要部署Dashboard指定有效的证书，才可以访问. 
# 谷歌浏览器必须要换证书，火狐可以忽略跳过去.

因为其他浏览器打不开是因为证书过期，我们可以解决证书过期问题

# 首先需要生成证书

API Server访问Dashboard(推荐)

# 如果Kubernetes API服务器是公开的，并可以从外部访问，我们可以直接使用API Server的方式来访问.  
# https://:/api/v1/namespaces/kube-system/services/https:kubernetes-dashboard:/proxy/  
# https://172.19.0.26:6443/api/v1/namespaces/kube-system/services/https:kubernetes-dashboard:/proxy/  
# 如果返回一下结果不要慌，这是因为新版的K8s默认启用了RBAC,并未认证用户赋予了一个默认的身份: anconymous  
# 对于API Server来说，他是使用证书进行认证的，我们需要创建一个证书  

# 1. 首先找到kubectl命令的配置文件，默认情况下为/etc/kubernetes/admin.conf  
# 2. 然后我们使用client-certificate-data和client-key-data生成一个*p12*文件，可使用下列命令：  
  
# 生成client-certificate-data   
grep 'client-certificate-data' ~/.kube/config | head -n 1 | awk '{print $2}' | base64 -d >> kubecfg.crt   
  
# 生成client-key-data   
grep 'client-key-data' ~/.kube/config | head -n 1 | awk '{print $2}' | base64 -d >> kubecfg.key   
  
# 生成p12   
openssl pkcs12 -export -clcerts -inkey kubecfg.key -in kubecfg.crt -out kubecfg.p12 -name "kubernetes-client"  
  
ls  
kubecfg.crt  kubecfg.key  kubecfg.p12  
# 最后导入上面生成的p12文件，重新打开浏览器，显示如下  

使用Token登录

kubectl create serviceaccount dashboard-admin -n kube-system
kubectl create clusterrolebinding dashboard-cluster-admin --clusterrole=cluster-admin --serviceaccount=kube-system:dashboard-admin
kubectl get secret -n kube-system |grep dashboard
dashboard-admin-token-drb8d                      kubernetes.io/service-account-token   3      2m43s
dashboard-cert                                   Opaque                                2      21m

kubectl describe secret dashboard-admin-token-drb8d -n kube-system

在创建一个账号，拥有对default名称空间的权限

kubectl create serviceaccount def-ns-admin -n default
kubectl create rolebinding def-ns-admin --clusterrole=admin --serviceaccount=default:def-ns-admin

kubectl get secret
NAME                       TYPE                                  DATA   AGE
def-ns-admin-token-mnpph   kubernetes.io/service-account-token   3      2m29s
default-token-qw5gz        kubernetes.io/service-account-token   3      18h
[root@master pki]# kubectl describe secret def-ns-admin-token-mnpph

配置文件认证

kubectl config set-cluster kubernetes --certificate-authority=./ca.crt --server="https://172.19.0.26:6443"  --embed-certs=true --kubeconfig=/root/def-ns-admin.conf
Cluster "kubernetes" set.
[root@master pki]# kubectl config view --kubeconfig=/roo/def-ns-admin.conf
apiVersion: v1
clusters: null
contexts: null
current-context: ""
kind: Config
preferences: {}
users: null
kubectl get secret def-ns-admin-token-mnpph -o json
DEF_NS_ADMIN_TOKEN=$(kubectl get secret def-ns-admin-token-mnpph -o jsonpath={.data.token} |base64 -d )
kubectl config set-credentials def-ns-admin --token=$DEF_NS_ADMIN_TOKEN --kubeconfig=/root/def-ns-admin.conf
kubectl config view --kubeconfig=/root/def-ns-admin.conf

kubectl config set-context def-ns-admin@kubernetes --cluster=kubernetes --user=def-ns-admin --kubeconfig=/root/def-ns-admin.conf

kubectl config use-context def-ns-admin@kubernetes --kubeconfig=/root/def-ns-admin.conf

kubectl config view --kubeconfig=/root/def-ns-admin.conf
# 将此文件上传到电脑目录上.