ceph-ansible 部署

利用ceph-ansible工程，快速部署ceph集群
Centos 7
ansible 2.7
ceph luminous
ceph-ansible stable-3.2

注意：
根据ceph-ansible github说明，可通过stable-*分支稳定部署ceph

stable-2.1 Support for ceph version jewel. This branch supports ansible versions 2.1 and 2.2.1.
stable-2.2 Support for ceph versions jewel and kraken. This branch supports ansible versions 2.1 and 2.2.2.
stable-3.0 Support for ceph versions jewel and luminous. This branch supports ansible versions 2.3.1, 2.3.2 and 2.4.2.
master Support for ceph versions jewel, and luminous. This branch supports ansible version 2.4.2

注意不同版本的ansible，需要使用不同的stable分支，最新分支为stable3.2，通过/ceph-ansible/roles/ceph-validate/tasks/check_system.yml中描述，stable3.2支持ansible版本为2.4.x-2.6.x

- name: fail on unsupported ansible version
  fail:
    msg: "Ansible version must be between 2.4.x and 2.6.x!"
  when:
    - ansible_version.major|int == 2
    - (ansible_version.minor|int < 4 or ansible_version.minor|int > 6)

这里需要使用ansible2.7操作ceph-ansible项目，使用分支为stable3.2，需要将上面代码注释掉，否则版本验证这里不会通过

创建主机3台，依次命名为mon, osd1, osd2
mon 运行mon，mgr服务
osd1，osd2只启动osd服务
三台主机配置mon到mon，osd1，osd2的root用户免密码ssh登录，注意mon到mon节点也要配置免密码ssh登录，配置主机名与节点名称相同，配置/etc/hosts

10.0.0.6 mon
10.0.0.4 osd1
10.0.0.3 osd2

确认网卡均为eth0，并检查挂载硬盘设备，例如/dev/vdb

安装ansible（mon节点执行）

yum -y install ansible
检查ansible版本

[root@mon ceph-ansible]# ansible --version
ansible 2.7.8
  config file = /root/ceph-ansible2/ceph-ansible/ansible.cfg
  configured module search path = [u'/root/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/lib/python2.7/site-packages/ansible
  executable location = /usr/bin/ansible
  python version = 2.7.5 (default, Jul 13 2018, 13:06:57) [GCC 4.8.5 20150623 (Red Hat 4.8.5-28)]

加载ceph-ansible源码（mon节点执行）

#git clone https://github.com/ceph/ceph-ansible.git
#git checkout stable-3.2

修改配置（mon节点执行）

修改/etc/ansible/hosts

[mons]
mon
[mgrs]
mon
[osds]
osd1
osd2

修改/ceph-ansible/group_vars/all.yml

#cp all.yml.sample all.yml
#vi all.yml
#cat all.yml
ceph_origin: repository
ceph_repository: community
ceph_stable_release: luminous
public_network: "10.0.0.0/24"
cluster_network: "10.0.0.0/24"
monitor_interface: eth0
devices:
  - '/dev/vdb'
osd_scenario: collocated

public_network,cluster_network为当前网段信息，monitor_interface配置网卡为各个主机网卡名称
devices配置为硬盘设备，ceph_stable_release配置ceph版本，这里使用luminous

修改/ceph-ansible/site.yml

#cp site.yml.sample site.yml

主要配置该文件hosts

- hosts:
  - mons
  #- agents
  - osds
  #- mdss
  #- rgws
  #- nfss
  #- restapis
  #- rbdmirrors
  #- clients
  - mgrs
  #- iscsigws
  #- iscsi-gws # for backward compatibility only!

这里只留下mons,osds,mgrs

安装ceph

#cd /ceph-ansible
#ansible-playbook site.yml

我这里安装一路顺利，如果有问题的话，可以用ansible-playbook site.yml -vvv再装一次，提示信息更详细，检查下有什么问题

安装完成后，检查ceph集群状态

#ceph -s
  cluster:
    id:     4ff55516-ade8-4801-b7bc-ad689bd75efd
    health: HEALTH_OK
 
  services:
    mon: 1 daemons, quorum mon
    mgr: mon(active)
    osd: 2 osds: 2 up, 2 in
 
  data:
    pools:   0 pools, 0 pgs
    objects: 0 objects, 0B
    usage:   2.00GiB used, 37.8GiB / 39.8GiB avail
    pgs:

添加dashboard

#ceph mgr module enable dashboard
#ceph mgr dump
{
    "epoch": 10,
    "active_gid": 4136,
    "active_name": "mon",
    "active_addr": "10.0.0.6:6800/18798",
    "available": true,
    "standbys": [],
    "modules": [
        "dashboard",
        "status"
    ],
    "available_modules": [
        "balancer",
        "dashboard",
        "influx",
        "localpool",
        "prometheus",
        "restful",
        "selftest",
        "status",
        "zabbix"
    ],
    "services": {
        "dashboard": "http://mon:7000/"
    }
}

这里可以看到，dashboard直接启动了，并且7000端口的web service也成功启动了