1. 准备机器, 1台做ansible, 另外3台安装ceph
http://docs.ceph.com/ceph-ansible/master/#demos
2. 在安装了ansible的机器上, vi /etc/hosts. 加三台机器的hostname
10.0.221.100 ceph1
10.0.221.101 ceph2
10.0.221.102 ceph3
3. 添加秘钥
ssh-keygen -t rsa
ssh-copy-id root@ceph1
ssh-copy-id root@ceph2
ssh-copy-id root@ceph3
4. 安装ceph-ansible
git clone https://github.com/ceph/ceph-ansible.git
pip install -r requirements.txt
5. 配置文件设置
vi nodes.yml
[mons]
10.0.221.100
10.0.221.101
10.0.221.102
[osds]
10.0.221.100
10.0.221.101
10.0.221.102
[mgrs]
10.0.221.100
10.0.221.101
10.0.221.102
==========================================
cp group_vars/all.yml.sample group_vars/all.yml
vi group_vars/all.yml
cluster: ceph #集群名
ceph_origin: repository #使用distro,则不会使用公网源
ceph_repository: community #被操作节点使用本地的repo文件
ceph_stable_release: luminous #安装版本
public_network: "10.0.221.0/24"
cluster_network: "10.0.221.0/24"
mon_host: 10.0.221.100,10.0.221.101,10.0.221.102
monitor_interface: ens160
devices: #OSD磁盘,有几块写几块
- '/dev/sdb'
osd_scenario: collocated
========================================
cp site.yml.sample site.yml
6. ceph 安装
ansible-playbook site.yml -i nodes.yml
7. ceph -s 检查
如果osd未up,
chown ceph:ceph /dev/sdb1
chown ceph:ceph /dev/sdb2
systemctl restart [email protected]
如果restart有问题,
systemctl reset-failed [email protected] 再 restart
8 检查
ceph osd tree
ceph -s
ceph osd pool create kube 128 128
ceph osd pool ls
ceph osd pool ls detail
netstat -anp | grep 6789
rbd list kube
rbd create kube/vol1 --size 1000
9. ceph dashboard, 在ceph mon的机器上
ceph mgr module enable dashboard
netstat -antpl | grep ceph-mgr|grep LISTE
access http://10.0.221.100:7000
10. enable ceph-rest-api,在ceph mon的机器上
ceph-rest-api -n client.admin &
netstat -anp | grep 5000
https://ceph.com/category/ceph-rest-api/
https://blog.csdn.net/hit1944/article/details/46551753
可以通过5000 访问restapi.
11. 一个基于ceph-rest-api的dashboard
docker run -d --net=host tobegit3hub/ceph-web
access http://10.0.221.100:8080/
12. openattic存储管理,安装openattic
sudo add-apt-repository "deb http://apt.openattic.org/ $(lsb_release -cs) main"
sudo apt-get install openattic
sudo apt-get install openattic-gui
oaconfig install
apt-get install openattic-module-drbd
mkdir /etc/ceph
cd /etc/ceph
copy /etc/ceph/ceph.conf and /etc/ceph/ceph.client.admin.keyring to this dir from ceph mon node
chgrp openattic /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring
chmod g+r /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring
apt-get install openattic-module-ceph
apt-get install openattic-filter
vi /etc/dbconfig-common/openattic.conf
vi ceph.conf
oaconfig install
access http://10.0.221.103/openattic
reference:
https://docs.openattic.org/en/latest/install_guides/accessing_the_webui.html
https://blog.csdn.net/qq_23348071/article/details/76222955
13, 安装inkscope, ceph dashboard, 需要安装到ceph mon node上
git clone https://github.com/inkscope/inkscope-ansible.git
cd inkscope-ansible/
ansible-galaxy install -r requirements.yaml
vi host
[inkscope]
10.0.221.100
vi inkscope.yaml
vi vars/playbook.yaml
vi vars/my_inkscope.yaml
ansible-playbook inkscope.yaml -i host
access http://10.0.221.100/inkscopeViz
refer to https://github.com/inkscope/inkscope-ansible
https://github.com/inkscope/inkscope
14. k8s 接ceph:
https://www.cnblogs.com/chimeiwangliang/p/8386353.html
15. k8s的rook(ceph)
https://blog.csdn.net/liukuan73/article/details/84664922
https://blog.csdn.net/hxpjava1/article/details/80175238