Kubernetes 之 StorageClass 资源(NFS)

Kubernetes 之 StorageClass 资源(NFS)

官网介绍文档：https://kubernetes.io/zh-cn/docs/concepts/storage/storage-classes/

Kubernetes 不包含内部 NFS 驱动。需要使用外部驱动为 NFS 创建 StorageClass。 这里有些例子：

• NFS Ganesha 服务器和外部驱动
• NFS subdir 外部驱动

本篇文档具体介绍NFS subdir外部驱动的操作方式，NFS Provisioner 是一个自动配置卷程序，它使用现有的和已配置的 NFS 服务器来支持通过持久卷声明动态配置 Kubernetes 持久卷。

1. 安装NFS Server端

# Centos/Redhat
$ yum install nfs-utils rpcbind -y
# debian/ubuntu
$ apt-get install nfs-kernel-server -y

2. NFS 共享内容如下

#IP: 192.168.5.44
#存储目录：/mnt/data

$ echo '/mnt/data  192.168.5.0/24(rw,sync,no_root_squash)' >> /etc/exports

$ exportfs -r

3. 获取项目所需要的yaml文件

$ git clone https://github.com/kubernetes-sigs/nfs-subdir-external-provisioner.git

$ mv nfs-subdir-external-provisioner/deploy  ~/nfs-provisioner

4. 修改并创建ServiceAccount

nfs-provisioner-rbac.yaml

kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: nfs-client-provisioner-runner
rules:
- apiGroups: [""]
  resources: ["persistentvolumes"]
  verbs: ["get", "list", "watch", "create", "delete"]
- apiGroups: [""]
  resources: ["persistentvolumeclaims"]
  verbs: ["get", "list", "watch", "update"]
- apiGroups: [""]
  resources: ["endpoints"]
  verbs: ["get", "list", "watch", "create", "update", "patch"]
- apiGroups: ["storage.k8s.io"]
  resources: ["storageclasses"]
  verbs: ["get", "list", "watch"]
- apiGroups: [""]
  resources: ["events"]
  verbs: ["create", "update", "patch"]
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: run-nfs-client-provisioner
subjects:
- kind: ServiceAccount
  name: nfs-client-provisioner
  namespace: >
roleRef:
  kind: ClusterRole
  name: nfs-client-provisioner-runner
  apiGroup: rbac.authorization.k8s.io
---
kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: leader-locking-nfs-client-provisioner
rules:
- apiGroups: [""]
  resources: ["endpoints"]
  verbs: ["get", "list", "watch", "create", "update", "patch"]
---
kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: leader-locking-nfs-client-provisioner
subjects:
- kind: ServiceAccount
  name: nfs-client-provisioner
  # replace with namespace where provisioner is deployed
  namespace: >
roleRef:
  kind: Role
  name: leader-locking-nfs-client-provisioner
  apiGroup: rbac.authorization.k8s.io

创建RBAC

$ kubectl apply -f nfs-provisioner-rbac.yaml 
clusterrole.rbac.authorization.k8s.io/nfs-client-provisioner-runner created
clusterrolebinding.rbac.authorization.k8s.io/run-nfs-client-provisioner created
role.rbac.authorization.k8s.io/leader-locking-nfs-client-provisioner created
rolebinding.rbac.authorization.k8s.io/leader-locking-nfs-client-provisioner created

kubectl create ns greatmap 创建命名空间

5. 部署 NFS Provisioner

nfs-provisioner-deploy.yaml

apiVersion: v1
kind: ServiceAccount
metadata:
  name: nfs-client-provisioner
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: nfs-client-provisioner
  labels:
    app: nfs-client-provisioner
  # replace with namespace where provisioner is deployed
  namespace: >
spec:
  replicas: 1
  strategy:
    type: Recreate
  selector:
    matchLabels:
      app: nfs-client-provisioner
  template:
    metadata:
      labels:
        app: nfs-client-provisioner
    spec:
      serviceAccountName: nfs-client-provisioner
      containers:
        - name: nfs-client-provisioner
          image: quay.io/external_storage/nfs-client-provisioner:latest
          #image: registry.cn-beijing.aliyuncs.com/pylixm/nfs-subdir-external-provisioner:v4.0.0  #国内镜像
          volumeMounts:
            - name: nfs-client-root
              mountPath: /persistentvolumes
          env:
            - name: PROVISIONER_NAME
              value: fuseim.pri/ifs  #外部制备器名称,根据情况修改
            - name: NFS_SERVER
              value: 192.168.5.44          # NFS服务器地址
            - name: NFS_PATH
              value: /mnt/data             # NFS共享的目录
      volumes:
        - name: nfs-client-root
          nfs:
            server: 192.168.5.44      # NFS服务器地址
            path: /mnt/data           # NFS共享的目录

创建 NFS Provisioner

$ kubectl apply -f nfs-provisioner-deploy.yaml
serviceaccount/nfs-client-provisioner created
deployment.apps/nfs-client-provisioner created

6. 创建 StorageClass

nfs-provisioner-class.yaml

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: nfs-client
provisioner: fuseim.pri/ifs  # 外部制备器提供者，需要与deploy中保持一致
parameters:
  archiveOnDelete: "false"  # 是否存档，false表示不存档，会删除oldPath下面的数据，true表示存档，会重命名路径
reclaimPolicy: Retain    # 回收策略，默认为Delete，可配置为Retain
volumeBindingMode: Immediate   # 默认为Immediate，表示创建PVC立即进行绑定

创建 StorageClass

$ kubectl apply -f nfs-provisioner-class.yaml 
storageclass.storage.k8s.io/nfs-client created

7. 测试

7.1 创建PVC测试

test-claim.yaml

kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: test-claim
spec:
  storageClassName: nfs-client    # 需要与StorageClass的名称保持一致
  accessModes:
    - ReadWriteMany
  resources:
    requests:
      storage: 10Mi

创建PVC

$ kubectl apply -f test-claim.yaml [-n greatmap]
persistentvolumeclaim/test-claim created

查看绑定状态

$ kubectl get pv    # 查看PV
NAME                                       CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS   CLAIM                 STORAGECLASS   REASON   AGE
pvc-e35db2a9-f893-423f-a8fb-5c2e563fa087   10Mi       RWX            Retain           Bound    greatmap/test-claim   nfs-client              2m7s
$ kubectl get pvc   # 查看PVC
NAME         STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS   AGE
test-claim   Bound    pvc-d298bed4-f360-4dd7-ae1c-c86882cf770f   10Mi       RWX            nfs-client     3m25s

# 状态为Bound，且PVC的VOLUME与PV的NAME一致，PV的CLAIM包含PVC的NAME，属于是互相绑定的状态。

7.2 创建Pod并绑定PVC测试

test-pod.yaml

kind: Pod
apiVersion: v1
metadata:
  name: test-pod
spec:
  containers:
  - name: test-pod
    image: busybox:stable
    command:
      - "/bin/sh"
    args:
      - "-c"
      - "touch /mnt/SUCCESS && exit 0 || exit 1"
    volumeMounts:
      - name: nfs-pvc
        mountPath: "/mnt"
  restartPolicy: "Never"
  volumes:
    - name: nfs-pvc
      persistentVolumeClaim:
        claimName: test-claim     # 需要与上面PVC的名称保持一致

创建Pod

$ kubectl apply -f test-pod.yaml [-n greatmap]
pod/test-pod created

进入NFS 服务器查看是否创建SUCCESS

$ ls default-test-claim-pvc-d298bed4-f360-4dd7-ae1c-c86882cf770f/
SUCCESS

# 持久卷被配置为：Namespace−{pvcName}-${pvName}