Oracle Container Engine for Kubernetes（OKE）上使用Dynamic Volume Provisioning

前提：

a,需要使用OCI（Oracle Cloud Infrastructure）Volume Provisioner：https://github.com/oracle/oci-volume-provisioner

b,仅支持OCI（Oracle Cloud Infrastructure）的Block Volumes

1，创建config.yaml，设置认证等信息。示例如下：

auth:
  tenancy: ocid1.tenancy.oc1..aaaaaaaatyn7scrtwt...
  user: ocid1.user.oc1..aaaaaaaao235lbcxvdrrqlr...
  key: |
    -----BEGIN RSA PRIVATE KEY-----
    MIIEowIBAAKCAQEUjVBnOgC4wA3j6CeTc6hIA9B3iwuJKyR8i7w...
    -----END RSA PRIVATE KEY-----
  passphrase: supersecretpassphrase
  fingerprint: 4d:f5:ff:0e:a9:10:e8:5a:d3:52:6a:f8:1e:99:a3:47
  region: us-phoenix-1

2，发布为Kubernetes Secret。

kubectl create secret generic oci-volume-provisioner \
    -n kube-system \
    --from-file=config.yaml=config.yaml

3，发布OCI Volume Provisioner。

kubectl apply -f oci-volume-provisioner-rbac.yaml
kubectl apply -f oci-volume-provisioner.yaml
kubectl apply -f storage-class.yaml
kubectl apply -f storage-class-ext3.yaml

各yaml文件内容如下

a,oci-volume-provisioner-rbac.yaml

---
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
  name: oci-provisioner-runner
rules:
  - apiGroups: [""]
    resources: ["persistentvolumes"]
    verbs: ["get", "list", "watch", "create", "delete"]
  - apiGroups: [""]
    resources: ["persistentvolumeclaims"]
    verbs: ["get", "list", "watch", "update", "create"]
  - apiGroups: ["storage.k8s.io"]
    resources: ["storageclasses"]
    verbs: ["get", "list", "watch"]
  - apiGroups: [""]
    resources: ["nodes"]
    verbs: ["list", "watch"]
  - apiGroups: [""]
    resources: ["events"]
    verbs: ["list", "watch", "create", "update", "patch"]
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
  name: run-oracle-provisioner
subjects:
  - kind: ServiceAccount
    name: oci-volume-provisioner
    namespace: kube-system
roleRef:
  kind: ClusterRole
  name: oci-provisioner-runner
  apiGroup: rbac.authorization.k8s.io
---
kind: ServiceAccount
apiVersion: v1
metadata:
  name: oci-volume-provisioner
  namespace: kube-system 

b,oci-volume-provisioner.yaml

apiVersion: apps/v1beta1
kind: Deployment
metadata:
  name: oci-volume-provisioner
  namespace: kube-system
spec:
  replicas: 1
  template:
    metadata:
      labels:
        app: oci-volume-provisioner
    spec:
      serviceAccountName: oci-volume-provisioner
      containers:
        - name: oci-volume-provisioner
          image: fra.ocir.io/oracle/oci-volume-provisioner:0.10.0
          env:
            - name: NODE_NAME
              valueFrom:
                fieldRef:
                  fieldPath: spec.nodeName
            - name: PROVISIONER_TYPE
              value: oracle.com/oci
          volumeMounts:
            - name: config
              mountPath: /etc/oci/
              readOnly: true
      volumes:
        - name: config
          secret:
            secretName: oci-volume-provisioner

c,storage-class.yaml

kind: StorageClass
apiVersion: storage.k8s.io/v1beta1
metadata:
  name: oci
provisioner: oracle.com/oci

d,storage-class-ext3.yaml

kind: StorageClass
apiVersion: storage.k8s.io/v1beta1
metadata:
  name: oci-ext3
provisioner: oracle.com/oci
parameters:
  fsType: ext3

确认storageclass

kubectl get sc

结果

NAME                     PROVISIONER          AGE
oci (default)            oracle.com/oci       81m
oci-ext3                 oracle.com/oci       81m

oci为默认storageclass。如果不是，使用下面命令把oci修改为默认storageclass。

kubectl patch storageclass oci -p '{"metadata": {"annotations":{"storageclass.kubernetes.io/is-default-class":"true"}}}'

 

4，创建一个pvc

kubectl apply -f nginx-volume.yaml

a,nginx-volume.yaml内容

kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: nginx-volume
spec:
  storageClassName: "oci"
  selector:
    matchLabels:
      failure-domain.beta.kubernetes.io/zone: "EU-FRANKFURT-1-AD-3"
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 50Gi

5，确认pvc

kubectl get pvc

结果如下

NAME           STATUS   VOLUME                                                                                         CAPACITY   ACCESS MODES   STORAGECLASS   AGE
nginx-volume   Bound    ocid1.volume.oc1.eu-frankfurt-1.xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx   50Gi       RWO            oci            27s

6，发布一个pod验证

kubectl apply -f nginx-pod.yaml

a,nginx-pod.yaml内容

kind: Pod
apiVersion: v1
metadata:
  name: nginx
spec:
  volumes:
    - name: nginx-storage
      persistentVolumeClaim:
        claimName: nginx-volume
  containers:
    - name: nginx
      image: nginx
      ports:
        - containerPort: 80
      volumeMounts:
      - mountPath: "/usr/share/nginx/html"
        name: nginx-storage

7，确认pod

kubectl get pod

结果如下

NAME                                READY   STATUS    RESTARTS   AGE
nginx                               1/1     Running   0          87s

其他注意事项，

a,如果block volume与kubernetes的node不是在同一个AD的话，发生了bound失败的问题。所以，作为学习，暂时还是在同一个AD里使用。

b,仅支持ReadWriteOnce模式，不支持ReadWriteMany。（ERROR：invalid access mode ReadWriteMany specified. Only ReadWriteOnce is supported）

结尾！