kubernetes部署openebs

openebs简介

OpenEBS是一款使用Go语言编写的基于容器的块存储开源软件。OpenEBS使得在容器中运行关键性任务和需要数据持久化的负载变得更可靠。

OpenEBS由CloudByte研发，这是一家专业做容器化存储的公司，OpenEBS是其一款开源产品，CloudByte将其在企业级容器存储的经验付诸到该项目中。这个项目的愿景也很简单，就是让需要持久化存储的工作负载中的存储服务能够直接集成在环境中，存储服务可以自动管理，将存储的细节隐藏起来，就像存储系统是另一套基础架构一样。

AWS中提供了EBS（Elastic Block Storage），适用于 Amazon EC2 的持久性块存储，可以满足要求最苛刻的应用程序在功能和性能方面的要求，OpenEBS即其开源实现。

官网：https://openebs.io/

项目地址：https://github.com/openebs/openebs

安装环境: 1个master节点（可调度），2个worker节点。

安装iscsi

OpenEBS要求配置iSCSI客户端，并且在worker nodes上运行iscsid服务

yum -y install iscsi-initiator-utils
systemctl enable --now iscsid

operator方式安装

部署operator

wget https://openebs.github.io/charts/openebs-operator.yaml
sed -i 's#quay.io#docker.io#g' openebs-operator.yaml
kubectl apply -f openebs-operator.yaml

查看创建的pod，其中ndm以daemonset方式部署：

[root@master01 ~]# kubectl -n openebs get pods
NAME                                          READY   STATUS    RESTARTS   AGE
maya-apiserver-864c5d5b95-nfdtn               1/1     Running   0          16m
openebs-admission-server-6c9d67888c-n2rgp     1/1     Running   0          16m
openebs-localpv-provisioner-88cb5c87d-dhpzl   1/1     Running   0          16m
openebs-ndm-2mwm4                             1/1     Running   0          16m
openebs-ndm-2snsc                             1/1     Running   0          16m
openebs-ndm-955bx                             1/1     Running   0          16m
openebs-ndm-operator-5b756b6f7b-jrvkd         1/1     Running   0          16m
openebs-provisioner-7fb6d4f999-d275b          1/1     Running   0          16m
openebs-snapshot-operator-8fb844448-vlsmm     2/2     Running   0          16m

查看创建的storage classes，openebs提供了4种类型storageclass：

[root@master01 ~]# kubectl get sc
NAME                        PROVISIONER                                                RECLAIMPOLICY   VOLUMEBINDINGMODE      ALLOWVOLUMEEXPANSION   AGE
openebs-device              openebs.io/local                                           Delete          WaitForFirstConsumer   false                  2d20h
openebs-hostpath            openebs.io/local                                           Delete          WaitForFirstConsumer   false                  2d20h
openebs-jiva-default        openebs.io/provisioner-iscsi                               Delete          Immediate              false                  2d20h
openebs-snapshot-promoter   volumesnapshot.external-storage.k8s.io/snapshot-promoter   Delete          Immediate              false                  2d20h

为了对OpenEBS进行简单测试，可以使用以下默认存储类

• openebs-jiva-default用于配置Jiva Volume（这使用default池，这意味着在Jiva副本容器的/ mnt / openebs_disk目录中创建了数据副本）
• openebs-hostpath 用于在主机路径上配置本地PV。
• openebs-device 用于在设备上配置本地PV。

也可以自定义创建新的storageclass

wget https://raw.githubusercontent.com/openebs/openebs/master/k8s/openebs-storageclasses.yaml
kubectl apply -f openebs-storageclasses.yaml

要使用实际磁盘，您必须根据需求创建cStorPools或Jiva池或OpenEBS Local PV，然后创建相应的StorageClass或使用默认StorageClass来使用它们。

[root@master01 ~]# kubectl -n openebs get blockdevice
NAME                                           NODENAME   SIZE          CLAIMSTATE   STATUS   AGE
blockdevice-3b1ce12540e21c97a36e7ee27723be30   node02     21474836480   Unclaimed    Active   23m
sparse-3dcb52e66513ae5739eac04d6ff737db        master01   10737418240   Unclaimed    Active   23m
sparse-c5fe83076c942cc743c1d9db3732f449        node01     10737418240   Unclaimed    Active   23m
sparse-f9f6728649ba0f6ec8b635128fc9d1d2        node02     10737418240   Unclaimed    Active   23m

[root@master01 ~]# kubectl  get  sp
NAME      AGE
default   2d21h

jiva示例

创建pvc

cat > openebs-nginx-demo.yaml <apiVersion: v1
kind: PersistentVolumeClaim
metadata:
 name: openebs1
spec:
 storageClassName: openebs-jiva-default
 accessModes:
  - ReadWriteOnce
 resources:
   requests:
     storage: 5Gi
---
apiVersion: v1
kind: Pod
metadata:
  name: nginx-pod1
  labels:
    name: nginx-pod1
spec:
  containers:
  - name: nginx-pod1
    image: nginx:alpine
    ports:
    - name: web
      containerPort: 80
    volumeMounts:
    - name: openebs1-vol1
      mountPath: /usr/share/nginx/html
  volumes:
  - name: openebs1-vol1
    persistentVolumeClaim:
      claimName: openebs1
EOF

部署示例

kubectl apply -f openebs-nginx-demo.yaml

查看自动创建的pv

[root@master01 ~]# kubectl get pv
NAME                                       CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS   CLAIM              STORAGECLASS           REASON   AGE
pvc-58a89132-5b2c-40b0-bfe3-32ac4d91d83e   5Gi        RWO            Delete           Bound    default/openebs1   openebs-jiva-default            15s


[root@master01 ~]# kubectl get pvc
NAME                                         STATUS    VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS           AGE
elasticsearch-data-quickstart-es-default-0   Pending                                                                                               37m
openebs1                                     Bound     pvc-58a89132-5b2c-40b0-bfe3-32ac4d91d83e   5Gi        RWO            openebs-jiva-default   17s

[root@master01 ~]# kubectl get pods
NAME                       READY   STATUS    RESTARTS   AGE
nginx-pod1                 1/1     Running   0          21s

local pv示例

对于不需要存储级复制的那些应用程序，LocalPV可能是一个不错的选择，因为它可以提供更高的性能。OpenEBS LocalPV与Kubernetes LocalPV相似，只不过它是由OpenEBS控制平面动态提供的，就像其他常规PV一样。OpenEBS LocalPV有两种类型hostpath-LocalPV或deviceLocalPV。hostpathLocalPV指主机上的子目录，而deviceLocalPV指节点上已发现的磁盘（直接连接或网络连接）。OpenEBS引入了LocalPV配置器，用于根据PVC和存储类规范中的某些标准选择匹配的磁盘或主机路径。有关OpenEBS LocalPV的更多详细信息，请参见此处。

主机路径配置的默认StorageClass名称是openebs-hostpath。默认主机路径配置为

/var/openebs/local

创建示例

cat > openebs-nginx-demo1.yaml <apiVersion: v1
kind: PersistentVolumeClaim
metadata:
 name: openebs-hostpath
spec:
 storageClassName: openebs-hostpath
 accessModes:
  - ReadWriteOnce
 resources:
   requests:
     storage: 5Gi
---
apiVersion: v1
kind: Pod
metadata:
  name: nginx-pod2
  labels:
    name: nginx-pod2
spec:
  containers:
  - name: nginx-pod2
    image: nginx:alpine
    ports:
    - name: web
      containerPort: 80
    volumeMounts:
    - name: openebs-hostpath
      mountPath: /usr/share/nginx/html
  volumes:
  - name: openebs-hostpath
    persistentVolumeClaim:
      claimName: openebs-hostpath
EOF

部署示例

kubectl apply -f openebs-nginx-demo1.yaml

查看创建的资源

[root@master01 ~]# kubectl get pv
NAME                                       CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS   CLAIM                      STORAGECLASS           REASON   AGE
pvc-58a89132-5b2c-40b0-bfe3-32ac4d91d83e   5Gi        RWO            Delete           Bound    default/openebs1           openebs-jiva-default            96s
pvc-80a7a25d-5c52-47b3-bbf7-38b15a28b189   5Gi        RWO            Delete           Bound    default/openebs-hostpath   openebs-hostpath                4s

[root@master01 ~]# kubectl get pvc
NAME                                         STATUS    VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS           AGE
openebs-hostpath                             Bound     pvc-80a7a25d-5c52-47b3-bbf7-38b15a28b189   5Gi        RWO            openebs-hostpath       8s
openebs1                                     Bound     pvc-58a89132-5b2c-40b0-bfe3-32ac4d91d83e   5Gi        RWO            openebs-jiva-default   98s

[root@master01 ~]# kubectl get pods
NAME                       READY   STATUS    RESTARTS   AGE
nginx-pod1                 1/1     Running   0          103s
nginx-pod2                 1/1     Running   0          13s