k8s rbd provider

问题

创建好ceph集群后,在k8s是中创建好storageclass后.创建pvc居然不能自动创建pv.报如下错误

failed to create rbd image: executable file not found in $PATH, command outpu

解决

google一把发现是kube-controller 镜像中无法找到 rbd命令导致的. 官方有个2016的issue已经解决了这个问题 https://github.com/kubernetes/kubernetes/issues/38923
在kubernetes的孵化器项目中有external-storage 的部署说明.
https://github.com/kubernetes-incubator/external-storage

部署storageClass 的provider

部署很简单,安装[https://github.com/kubernetes-incubator/external-storage/tree/master/ceph/rbd/deploy/rbac](ceph rbd)的部署说明,创建一个deployment就好了.

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: rbd-provisioner
spec:
  replicas: 1
  strategy:
    type: Recreate
  template:
    metadata:
      labels:
        app: rbd-provisioner
    spec:
      containers:
      - name: rbd-provisioner
        image: "quay.io/external_storage/rbd-provisioner:latest"
        env:
        - name: PROVISIONER_NAME
          value: ceph.com/rbd
      serviceAccount: rbd-provisioner

上面的deployment的yaml里面的环境变量,PROVISIONER_NAME 的值就是storageClass 的provisioner的值.

创建storageClass

allowVolumeExpansion: false
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  annotations:
    storageclass.kubernetes.io/is-default-class: "true"
  labels:
    cattle.io/creator: norman
  name: rbd-storage-class
parameters:
  adminId: 
  adminSecretName: 
  adminSecretNamespace: 
  fsType: ext4
  imageFeatures: layering
  imageFormat: "2"
  monitors: 
  pool: kube
  userId: k8s
  userSecretName: 
provisioner: ceph.com/rbd
reclaimPolicy: Delete
volumeBindingMode: Immediate

配置好ceph的访问secret就可以正常使用ceph rbd 作为pv provisioner.