kubernetes上rook-ceph的部署

k8s环境信息:

主机名	操作系统	私有IP	可用存储设备
k8s-0001	CentOS 7.5 64bit	172.16.192.127
k8s-0002	CentOS 7.5 64bit	172.16.192.25
k8s-0003	CentOS 7.5 64bit	172.16.192.121
k8s-0004	CentOS 7.5 64bit	172.16.192.243	/dev/vdb
k8s-0005	CentOS 7.5 64bit	172.16.192.226	/dev/vdb

[root@k8s-0001 ~]# kubectl get nodes
NAME             STATUS    ROLES     AGE       VERSION
172.16.192.121   Ready     master    3d        v1.11.1
172.16.192.127   Ready     master    3d        v1.11.1
172.16.192.226   Ready     node      3d        v1.11.1
172.16.192.243   Ready     node      3d        v1.11.1
172.16.192.25    Ready     master    3d        v1.11.1

所有节点开启ip_forward:

cat <  /etc/sysctl.d/ceph.conf
net.ipv4.ip_forward = 1
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
EOF
sysctl --system

打标签

运行ceph-mon的节点打上：ceph-mon=enabled:

kubectl label nodes {172.16.192.243,172.16.192.226} ceph-mon=enabled

运行ceph-osd的节点，也就是存储节点，打上：ceph-osd=enabled:

kubectl label nodes {172.16.192.243,172.16.192.226} ceph-osd=enabled

运行ceph-mgr的节点打上：ceph-mgr=enabled

kubectl label nodes 172.16.192.243 ceph-mgr=enabled

部署operator:

git clone https://github.com/rook/rook.git
cd rook
git checkout -b release-0.9 remotes/origin/release-0.9
cd cluster/examples/kubernetes/ceph
# edit operator.yaml if you need
kubectl create -f operator.yaml

operator.zip

部署cluster:

kubectl apply -f cluster.yaml

cluster.zip

创建storageclass

kubectl apply -f storageclass.yaml
kubectl get storageclasses.storage.k8s.io -n rook-ceph

配置Rook Dashboard

cluster.yaml 中dashboard: enabled: true 配置为 true，就会自动创建 Dashboard 服务，但默认为ClusterIP服务类型。

kubectl get svc -n rook-ceph | grep mgr-dashboard
kubectl create -f dashboard-external-https.yaml
kubectl get svc -n rook-ceph | grep mgr-dashboard

访问登录页面

用户名为admin，获取密码：

kubectl -n rook-ceph get secret rook-ceph-dashboard-password -o jsonpath='{.data.password}'  |  base64 --decode